BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//ILCB - ECPv6.15.20//NONSGML v1.0//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-ORIGINAL-URL:https://www.ilcb.fr
X-WR-CALDESC:Events for ILCB
REFRESH-INTERVAL;VALUE=DURATION:PT1H
X-Robots-Tag:noindex
X-PUBLISHED-TTL:PT1H
BEGIN:VTIMEZONE
TZID:Europe/Paris
BEGIN:DAYLIGHT
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
TZNAME:CEST
DTSTART:20200329T010000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
TZNAME:CET
DTSTART:20201025T010000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
TZNAME:CEST
DTSTART:20210328T010000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
TZNAME:CET
DTSTART:20211031T010000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
TZNAME:CEST
DTSTART:20220327T010000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
TZNAME:CET
DTSTART:20221030T010000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTART;TZID=Europe/Paris:20211112T160000
DTEND;TZID=Europe/Paris:20211112T170000
DTSTAMP:20260411T224020
CREATED:20211002T073730Z
LAST-MODIFIED:20211018T094923Z
UID:13464-1636732800-1636736400@www.ilcb.fr
SUMMARY:CoCoDev
DESCRIPTION:BabyBERTa: Learning More Grammar With Small-Scale Child-Directed Language\n  Philip Huebner  (University of Illinos\, Urbana-Champaign) \nAbstract: \nTransformer-based language models have taken the NLP world by storm. However\, their potential for addressing important questions in language acquisition research has been largely ignored. In this work\, we examined the grammatical knowledge of RoBERTa (Liu et al.\, 2019) when trained on a 5M word corpus of language acquisition data to simulate the input available to children between the ages 1 and 6. Using the behavioral probing paradigm\, we found that a smaller version of RoBERTa-base that never predicts unmasked tokens\, which we term BabyBERTa\, acquires grammatical knowledge comparable to that of pre-trained RoBERTa-base – and does so with approximately 15X fewer parameters and 6\,000X fewer words. We discuss implications for building more efficient models and the learnability of grammar from input available to children. Lastly\, to support research on this front\, we release our novel grammar test suite that is compatible with the small vocabulary of child-directed input. \n 
URL:https://www.ilcb.fr/event/babyberta-learning-more-grammar-with-small-scale-child-directed-language/
LOCATION:via zoom
CATEGORIES:Seminars
END:VEVENT
END:VCALENDAR