Sep 19, 2023 · In this work, we introduce the Qwen-VL series, a set of large-scale vision-language models (LVLMs) designed to perceive and understand both texts and images. Starting from the Qwen-LM as a foundation, we endow it with visual capacity by the meticulously designed (i) visual receptor, (ii) input-output interface, (iii) 3-stage training pipeline ...

The overall network architecture of Qwen-VL consists of three components and the details of model parameters are shown in Table 1: Large Language Model: Qwen-VL adopts a large language model as its foundation component. The model is initialized with pre-trained weights from Qwen-7B (Qwen, 2023).

Published as a conference paper at ICLR 2024 3.1 LOSS OF HIGH FREQUENCY INFORMATION - "NTK-AWARE" INTERPOLATION If we look at rotary position embeddings (RoPE) only from an information encoding perspective,

Sep 12, 2024 · Our experiments demonstrate AIR's effectiveness across state-of-the-art LLMs, achieving an attack success rate (ASR) exceeding $\textbf{90}$% on most models, including GPT-4o, Claude-3.5-Sonnet, and Qwen-2-72B. Notably, we observe an inverse scaling phenomenon, where larger models are more vulnerable to this attack method.

Sep 26, 2024 · Additionally, we evaluate three categories of LLMs against this benchmark: 1) proprietary LLM services such as GPT-4; 2) public LLMs like QWen; and 3) specialized medical LLMs, like HuatuoGPT2. Through this extensive evaluation, we aim to provide a better understanding of LLMs' performance in the medical domain, ultimately contributing to their ...

Welcome to the OpenReview homepage for ICLR 2024

Building upon this formulation, the experts in both Qwen (Team, 2024) and Mixtral (Jiang et al., 2024) adopt the structure of LLaMA (Touvron et al., 2023). Specifically, the feed-forward network (FFN) within each expert consists of three linear layers that function as Eq. (2), where ⊙signifies element-wise multiplication,W up,W gate ∈Rd h× ...

Jan 16, 2024 · The aligned Large Language Models (LLMs) are powerful language understanding and decision-making tools that are created through extensive alignment with human feedback.

tuning phase. To lift the OCR accuracy and support other languages, e.g., Chinese, Qwen-VL Bai et al. (2023b) unfreezes its image encoder (a CLIP-G) and uses lots of OCR data in its stage-two training. Innovatively, Vary Wei et al. (2023) generates a new …

Qwen-VL-Chat SELF-FILTER-7B Qwen-VL-7B Libra-7B Figure 1: A comparison of TIVE-8B with other open-source models in terms of the instruction data scale and average benchmark performance on MME, SEED-Bench, MMBench, ScienceQA. To this end, in this paper, we propose a data selection approach for visual instruction tun-