Literature Manager | Candidate Papers

15/02/2023

136

169

Grégoire Mialon,Roberto Dessì,M. Lomeli,Christoforos Nalmpantis,Ramakanth Pasunuru,Roberta Raileanu,Baptiste Rozière,Timo Schick,Jane Dwivedi-Yu,Asli Celikyilmaz,Edouard Grave,Yann LeCun,Thomas Scialom

The missing token objective allows ALMs to learn to reason, use tools, and even act, while still performing standard natural language tasks and even outperforming most regular LMs on several benchmarks.

Complex QA and language models hybrid architectures, Survey

17/02/2023

346

Xavier Daull,P. Bellot,Emmanuel Bruno,Vincent Martin,Elisabeth Murisasco

Current solutions and promising strategies are reviewed, using elements such as hybrid LLM architectures, human-in-the-loop reinforcement learning, prompting adaptation, neuro-symbolic and structured knowledge grounding, program synthesis, and others, and an overview of the current research and trends in the area of complex QA.

A Survey of Deep Learning for Mathematical Reasoning

ArXiv

20/12/2022

216

Pan Lu,Liang Qiu,Wenhao Yu,S. Welleck,Kai-Wei Chang

This survey paper reviews the key tasks, datasets, and methods at the intersec-tion of mathematical reasoning and deep learning over the past decade, and evaluates existing benchmarks and methods and discusses future research directions.

Is ChatGPT a General-Purpose Natural Language Processing Task Solver?

08/02/2023

Chengwei Qin,Aston Zhang,Zhuosheng Zhang,Jiaao Chen,Michihiro Yasunaga,Diyi Yang

It is found that ChatGPT performs well on many tasks favoring reasoning capabilities while it still faces challenges when solving specific tasks such as sequence tagging, and with extensive empirical studies, both the effectiveness and limitations of the current version of ChatG PT are demonstrated.

Scaling Instruction-Finetuned Language Models

ArXiv

20/10/2022

104

Hyung Won Chung,Le Hou,S. Longpre,Barret Zoph,Yi Tay,W. Fedus,Eric Li,Xuezhi Wang,M. Dehghani,Siddhartha Brahma,Albert Webson,S. Gu,Zhuyun Dai,Mirac Suzgun,Xinyun Chen,Aakanksha Chowdhery,Dasha Valter,Sharan Narang,Gaurav Mishra,A. Yu,Vincent Zhao,Yanping Huang,Andrew M. Dai,Hongkun Yu,Slav Petrov,E. Chi,J. Dean,Jacob Devlin,Adam Roberts,Denny Zhou,Quoc Le,Jason Wei

This result shows that instruction and UL2 continued pre-training are complementary compute-eﬃcient methods to improve the performance of language models without increasing model scale.