In the dynamic landscape of Artificial Intelligence (AI) and Natural Language Processing (NLP), NetMind.ai has consistently emerged as a persistent pioneer of innovative research and development. Today, we are pleased to share two exciting pieces of news from our research team, marking significant milestones in our journey towards contributing to the global AI and NLP research community.
Research Paper 1: A New Dataset and Method for Creativity Assessment Using the Alternate Uses Task
We're thrilled to announce that our paper, “A New Dataset and Method for Creativity Assessment Using the Alternate Uses Task,” has been accepted by the conference IC2023: International Symposium on Intelligent Computers, Algorithms, and Applications. This paper is a testament to the collaborative spirit and expertise of our authors, Zheng Yuan and Hongyi Gu from NetMind.ai. Also, we're grateful to the immeasurable help provided by our esteemed collaborators: Luning Sun, from the psychometric centre in Cambridge judge business school and Rebecca Myers, a PhD student in Psychology at the Centre for Neuroscience in Education
This paper investigates automating creativity ratings for the Alternate Uses Task (AUT) incorporating contextual models using a newly collected dataset. Results demonstrate that supervised models effectively distinguish creative from non-creative responses, even with imbalanced data and across different prompts.
Research Paper 2: Evaluation Metrics in the Era of GPT-4: Reliably Evaluating Large Language Models on Sequence to Sequence Tasks
This paper has been accepted for publication at the renowned EMNLP 2023 conference. EMNLP is one of the most prestigious NLP conferences globally; it provides a platform where the brightest minds in the field converge to share their groundbreaking work. The conference itself is scheduled from 6-10 December this year in Singapore, where we have been invited to present our work. The first author is our NLP Research Engineer, Andrea Sottana. Co-authors include Bin Liang from NetMind's team in China, Kai Zou (CEO of NetMind.ai) and Zheng Yuan, who were both responsible for supervising the project.
Automatic evaluation metrics are becoming increasingly inadequate as they're not keeping up with the pace of development of generative models. In this paper, we improve the understanding of current models' performance by providing a hybrid evaluation on a range of open and closed-source generative LLMs on multiple NLP benchmarks, using both automatic and human evaluation. We also explore the potential of the recently released GPT-4 to act as an evaluator.
These achievements are not mere milestones but are reflective of the hard work, innovative spirit, and collaborative efforts of our team members and external collaborators.
At NetMind.ai, we believe that research is a collective endeavor. The acceptance of our papers into these prestigious conferences is a testament to our team’s commitment to contribute significantly to the global discourse in AI and NLP.
Stay tuned for more updates and insights from NetMind.ai as we continue to navigate through the enthralling world of AI research and development.