XLNet Paper Presentation at ICML 2019
Zhilin Yang presents the XLNet paper at ICML 2019, introducing a groundbreaking AI model that outperforms BERT in natural language understanding tasks.
Setting
Main presentation hall at the Long Beach Convention & Entertainment Center, filled with rows of seating and a large projection screen at the front. The hall is spacious with high ceilings and modern decor.
Characters
The figures in this scene as an entity network — co-presence links everyone in the moment; speakers who trade lines are bound tighter. Turn the resolution dial to reveal depth the engine actually computed.
TNGF
SELECTED
Zhilin Yang
primary
A lean, young Chinese man in his late 20s with short, neatly styled black hair and rectangular glasses. His posture conveys both academic rigor and the energy of a rising star in machine learning research.
Conference Moderator
secondary
A middle-aged academic with a professional demeanor, average height with a slightly rounded posture from years of desk work. His dark hair is neatly combed, with streaks of gray at the temples. He wears rectangular wire-framed glasses that give him a studious appearance.
Senior Researcher
secondary
A middle-aged man with a lean build, sharp facial features, and short, neatly trimmed salt-and-pepper hair. His piercing eyes are framed by thin, rectangular glasses that reflect the projection screen's light. He has a slight furrow between his brows, indicating deep concentration.
Graduate Student
background
A young adult in their mid-20s with a lean build, slightly tousled dark brown hair, and wire-rimmed glasses perched on their nose. Their face is framed by a few stray curls that escaped from their casual ponytail, and their bright, inquisitive eyes dart between the speaker and their notebook.
Dialog
Conference Moderator
Without further ado, it's my honor to introduce Dr. Zhilin Yang, presenting groundbreaking work on XLNet—a model that rethinks autoregressive pretraining for NLP.
Zhilin Yang
Thank you. Let me walk through how XLNet overcomes BERT's limitations—starting with permutation language modeling. (points to projection) Note the bidirectional context here—
Senior Researcher
(interrupting) Wait—how does your attention masking avoid target leakage during permutations? The ablation studies don’t isolate this—
Zhilin Yang
(smiles) Excellent question. See appendix C—we introduce segment recurrence with memory units, unlike Transformer-XL’s implementation. Here’s the empirical comparison—
Senior Researcher
(muttering while scribbling) Hmph. Still seems like an overfit risk on short sequences with that attention span…
Zhilin Yang
To preempt further questions—yes, we’ve open-sourced the code. All hyperparameters are in section 4. Now, the SQuAD 2.0 results—(click)—these ROUGE scores speak for themselves.
Conference Moderator
(raising hand) We’ll hold Q&A until after the full presentation. Zhilin, proceed when ready.
Chat with Characters
Causal neighbors · 93 linked moments
J
1958
· same figure
2
2015
· same figure
B
2019
· same era
A
2017
· same era
A
2017
· precedes
A
2017
· same figure
F
2015
· same era
M
2015
· same era
G
2015
· same era
G
2015
· precedes
F
2016
· same era
F
2016
· precedes
F
2015
· precedes
M
2015
· precedes
L
2009
· same era
L
2009
· precedes
5
2010
· same era
2
2010
· same era
2
2010
· precedes
5
2010
· precedes
M
2019
· same era
M
2019
· precedes
2018
· same era
2018
· precedes
M
2014
· same era
A
2014
· same era
G
2016
· same era
M
2014
· follows
A
2014
· follows
G
2016
· follows
J
1958
· same figure
2
2015
· same figure
A
2017
· same figure
D
1960
· same figure
P
2018
· same figure
P
2017
· same figure