Hello there 👋
I am an incoming Research Scientist at Google AI in London, UK. Previously, I did my Ph.D. at the Institute for Logic, Language and Computation (ILLC) of the University of Amsterdam and at the School of Informatics of the University of Edinburgh. I was a part of the EdinburghNLP and AmsterdamNLP groups.
I also interned at Huggingface and Facebook AI Research (FAIR) in London, UK and at Amazon Research in Berlin, Germany.
My work focuses on Machine Reading Comprehension, also known as Question Answering. In particular, I am working on neural models that retrieve information from a collection of documents and then answer complex questions. I also work on Entity Linking and Entity Disambiguation. More generally, I am interested in (semi-)supervised and unsupervised deep neural network applications in combination with reasoning and reinforcement methods to approach Natural Language Understanding.
About
Download a PDF copy of my CV/Resume here.
Experience
present
London, UK
Fundamental research in Deep Learning for Search Engines and Question Answering algorithms.
Jul 2022
London, UK
Fundamental research in sparse retrieval system in combination with large language models.
Feb 2021
London, UK
Fundamental research in entity linking and retrieval using large-scale (multilingual) generative language models that led to two publications.
Sept 2019
July 2018
Amsterdam, Netherlands
Developing a generative adversarial network formulation for molecular graph prediction (de novo drug discovery) and developing variational auto-encoders in non-euclidean latent spaces (i.e., hyper-spheres and rotation lie groups). That led to three publications.
Jul 2016
Padua, Italy
Developed a supporting software tool and performed research in the area of botnet’s detection for the SPRITZ Security and Privacy Research Group.
Jul 2018
Schio, Italy
Full-stack web developer and database administrator.
Education
Sept 2022
Edinburgh, UK
Machine Learning and Deep Learning for Natural Language Processing. Worked on interpretable and controllable language models, graph-based questions answering, entity liking, and probabilistic models.
Sept 2018
Amsterdam, Netherlands
9/10 Cum Laude (top 2% national)
Sept 2016
Skills
-
Programming/markup Languages: Proficient in Python, Java, C/C++, LaTeX
Also ability with Matlab, SQL, PHP, JavaScript, HTML, .NET -
Technological Skills: Proficient in Pytorch, Tensorflow, NumPy/SciPy, Matplolib
Also ability with Pandas, Caffe2, Theano, Scikit-learn - Research Skills: Good mathematical background, writing skills, mentoring students, works well in a team.
Publications
See my Google Scholar page for a full list and citation counts.
In chronological order:
GenIE: Generative Information Extraction
Martin Josifoski, Nicola De Cao, Maxime Peyrard, Robert West (2022). In Conference of the North American Chapter of the Association for Computational Linguistics (NAACL 2022)
Sparse Interventions in Language Models with Differentiable Masking
Nicola De Cao, Leon Schmid, Dieuwke Hupkes, Ivan Titov (2021). In arXiv preprint arXiv:2112.06837
Highly Parallel Autoregressive Entity Linking with Discriminative Correction
Nicola De Cao, Wilker Aziz, Ivan Titov (2021). In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP 2021) Oral
Editing Factual Knowledge in Language Models
Nicola De Cao, Wilker Aziz, Ivan Titov (2021). In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP 2021) Oral
Multilingual Autoregressive Entity Linking
Nicola De Cao, Ledell Wu, Kashyap Popat, Mikel Artetxe, Naman Goyal, Mikhail Plekhanov, Luke Zettlemoyer, Nicola Cancedda, Sebastian Riedel, Fabio Petroni (2022). In Transactions of the Association for Computational Linguistics (TACL)
NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned
Sewon Min, Jordan Boyd-Graber, Chris Alberti, Danqi Chen, Eunsol Choi, Michael Collins, Kelvin Guu, Hannaneh Hajishirzi, Kenton Lee, Jennimaria Palomaki, Colin Raffel, Adam Roberts, Tom Kwiatkowski, Patrick Lewis, Yuxiang Wu, Heinrich Küttler, Linqing Liu, Pasquale Minervini, Pontus Stenetorp, Sebastian Riedel, Sohee Yang, Minjoon Seo, Gautier Izacard, Fabio Petroni, Lucas Hosseini, Nicola De Cao, Edouard Grave, Ikuya Yamada, Sonse Shimaoka, Masatoshi Suzuki, Shumpei Miyawaki, Shun Sato, Ryo Takahashi, Jun Suzuki, Martin Fajcik, Martin Docekal, Karel Ondrej, Pavel Smrz, Hao Cheng, Yelong Shen, Xiaodong Liu, Pengcheng He, Weizhu Chen, Jianfeng Gao, Barlas Oguz, Xilun Chen, Vladimir Karpukhin, Stan Peshterliev, Dmytro Okhonko, Michael Schlichtkrull, Sonal Gupta, Yashar Mehdad, Wen-tau Yih (2021). In arXiv preprint arXiv:2101.00133
A Memory Efficient Baseline for Open Domain Question Answering
Gautier Izacard, Fabio Petroni, Lucas Hosseini, Nicola De Cao, Sebastian Riedel, Edouard Grave (2020). In arXiv preprint arXiv:2012.15156
Autoregressive Entity Retrieval
Nicola De Cao, Gautier Izacard, Fabio Petroni, Sebastian Riedel (2021). In Proceedings of the 9th International Conference on Learning Representations (ICLR) Spotlight (top 5%)
Interpreting Graph Neural Networks for NLP With Differentiable Edge Masking
Michael Sejr Schlichtkrull, Nicola De Cao, Ivan Titov (2021). In Proceedings of the 9th International Conference on Learning Representations (ICLR) Spotlight (top 5%)
KILT: a Benchmark for Knowledge Intensive Language Tasks
Fabio Petroni, Aleksandra Piktus, Angela Fan, Patrick Lewis, Majid Yazdani, Nicola De Cao, James Thorne, Yacine Jernite, Vassilis Plachouras, Tim Rocktäschel, Sebastian Riedel (2021). In Conference of the North American Chapter of the Association for Computational Linguistics (NAACL 2021).
The Power Spherical distribution
Nicola De Cao, Wilker Aziz (2020). In Proceedings of the 37th International Conference on Machine Learning (ICML 2020), INNF+.
How do Decisions Emerge across Layers in Neural Models? Interpretation with Differentiable Masking
Nicola De Cao, Michael Sejr Schlichtkrull, Wilker Aziz, Ivan Titov (2020). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020).
Block Neural Autoregressive Flow
Nicola De Cao, Wilker Aziz, Ivan Titov (2019). In 35th Conference on Uncertainty in Artificial Intelligence (UAI 2019).
Question Answering by Reasoning Across Documents with Graph Convolutional Networks
Nicola De Cao, Wilker Aziz, Ivan Titov (2019). In Conference of the North American Chapter of the Association for Computational Linguistics (NAACL 2019).
Explorations in Homeomorphic Variational Auto-Encoding
Luca Falorsi*, Pim de Haan*, Tim R. Davidson*, Nicola De Cao, Maurice Weiler, Patrick Forré, Taco S. Cohen (2018). In ICML 2018 workshop on Theoretical Foundations and Applications of Deep Generative Models.
*equal contribution.
MolGAN: An implicit generative model for small molecular graphs
Nicola De Cao, Thomas Kipf (2018). In ICML 2018 workshop on Theoretical Foundations and Applications of Deep Generative Models.
Hyperspherical Variational Auto-Encoders
Tim R. Davidson*, Luca Falorsi*, Nicola De Cao*, Thomas Kipf, Jakub M. Tomczak (2018). In 34th Conference on Uncertainty in Artificial Intelligence (UAI 2018). Spotlight
*equal contribution.
Deep Generative Models for Graphs: VAEs, GANs, and reinforcement learning for de novo drug discovery
Nicola De Cao, supervised by Thomas Kipf and Max Welling (2018). Master Thesis
Published at in ICML 2018 workshop on Theoretical Foundations and Applications of Deep Generative Models as MolGAN: An implicit generative model for small molecular graphs.
Book Me
I offer a consultation service for businesses:
- Technical advice on machine learning methods and algorithms
- Evaluating the feasibility of your product by analysing its technical challenges
- Advising on market needs and helping you define precise business goals
and for privates who want advice to get the job you want in tech:
- Specific advice to help you break into tech
- Build a career coaching or study plan to reach your goals
- Review your resume or cover letter
I charge hourly, so you do not need to engage with me more than needed!
Write to me at any moment to arrange what is the best time to meet.