WHAT IS IT ALL ABOUT?
GigaVerbo is the name of the dataset developed by the researchers. The project "Tucano: Advancing Neural Text Generation for Portuguese" aims to bridge the resource gap in Portuguese natural language processing (NLP) by providing high-quality datasets and cutting-edge language models specifically designed for the Portuguese language. The development and release of the GigaVerbo corpus, comprising 200 billion deduplicated tokens, along with the Tucano family of models, aims to foster progress in neural text generation in an open and reproducible manner, promoting equitable access.
HOW DID THEY PROCEED?
The researchers collected several Portuguese corpora from different sources to ensure high linguistic diversity and quality. These corpora were then deduplicated and filtered to form the GigaVerbo dataset. Using this dataset, they trained several decoder models on the Marvin supercomputer, which followed rigorous evaluation and optimization cycles.
WHAT GAP DOES THE PROJECT FILL?
The project addresses two major gaps: first, the scarcity of comprehensive open-source resources for Portuguese, a language often overshadowed by resource-rich languages like English. Second, the deficiency in open-source LLM development, which impedes the scientific reproducibility of these models.
HOW DID YOU USE THE MARVIN SUPERCOMPUTER?
The Marvin cluster was crucial for training the Tucano models. We leveraged its powerful computing capabilities to process the large GigaVerbo dataset efficiently, train the Tucano series, and conduct extensive evaluations using multiple benchmarks.
WHAT IS THE NEXT STEP?
The researchers are currently working to scale up their developments in Portuguese by improving their dataset and training larger models. They are also currently developing resources for other low-resource languages, such as Bengali and Hindi, all thanks to Marvin and the University of Bonn.
WHO WAS INVOLVED IN THE PROJECT?
Nicholas Kluge Correa (Center for Science and Thought), Aniket Sen (High Performance Computing and Analytics Lab and Helmholtz Institute for Radiation and Nuclear Physics), Sophia Falk (Institute for Science and Ethics), and Shiza Fatimah (Institute for Computer Science).
WHAT IS THE SOURCE?
Nicholas Kluge Corrêa, Aniket Sen, Sophia Falk, Shiza Fatimah: Tucano: Advancing Neural Text Generation for Portuguese, Patterns, DOI: 10.1016/j.patter.2025.101325
WHERE CAN I FIND OUT MORE?
Dr. Nicholas Kluge Correa, Transdisciplinary Research Unit “Sustainable Futures”, Institute of Philosophy, Center for Science and Thought, Tel. +49 (0)228/73-54017, E-Mail: kluge@uni-bonn.de, Internet: https://nkluge-correa.github.io/Tucano/