Establish benchmarks to assist researchers and validate AI-generated results
Since the introduction of artificial intelligence, the financial sector has witnessed significant layoffs, substantial market rallies, and a surge in demand for specialized expertise that is scarce. The rapid integration of AI into finance provides immediate insight into the benefits and potential repercussions that could ripple across the economy, as more industries adopt similar technologies.
This technology has also enabled finance researchers to tackle questions that were either impractical or impossible to address using traditional methods. AI significantly accelerates tasks like programming and statistical analysis, making them quicker and more cost-effective. However, the advantages of AI in research extend well beyond just handling routine tasks that are slow or unmanageable for humans.
In a working paper, Andrea Eisfeldt from UCLA and Gregor Schubert highlight various AI applications for finance researchers. These include simulating human survey responses, generating new research hypotheses based on findings, and analyzing corporate earnings sentiments. They urge researchers to leverage this new opportunity, partly by providing examples of studies that have used different AI techniques.
Verification is Key
However, there is a caveat. Researchers may not always know if they can rely on AI-generated results. The specific factors that could affect these advantageous features are still unknown, and improper use of AI analyses could skew study outcomes. Therefore, validating the accuracy of this powerful tool will likely necessitate numerous traditional testing methods.
Eisfeldt and Schubert encourage financing researchers to embrace this challenge for the greater good of their field. They advocate for the publication of studies that analyze how these AI tools perform in various contexts and how different design choices impact results, noting that many researchers will be employing AI for specific, novel purposes.
“The goal should possibly be to develop a repertoire of ‘canonical’ methods and tests,” they write. “Such tests would function similarly to standard diagnostic tools currently available for econometric methods, such as differences in differences. These diagnostic tools could reassure readers and reviewers about the reliability of AI-generated analyses.”
Assessing Initial Findings
Eisfeldt and Schubert begin their article by outlining their approach in a related working paper they co-authored with Miao Ben Zhang from USC and Bledi Taska from Skyhive during the early days following AI’s generative capabilities announcement.
In that earlier paper, the team constructed and compared stock portfolios made up of companies across various industries—those highly exposed to AI (likely to benefit significantly) versus those with low exposure. They also developed a novel method to evaluate whether AI is expected to augment a profession or replace human labor.
Portfolios with high AI exposure outperformed the lowest 5% of portfolios, achieving an average daily gain of 0.45% within two weeks after the launch of ChatGPT on November 30. These initial gains were primarily observed in companies poised to use AI for job replacement (for additional insights on these findings, refer to this REVIEW article).
The document by Eisfeldt and Schubert further explores how various AI applications have been employed in finance research and education. For instance, text classification can assess sentiment from telephone transcriptions and other corporate communications, while information retrieval can streamline the process of analyzing vast amounts of documents to address specific queries related to businesses or regulations. The paper also presents new ideas for potential AI applications in future research—indicative of the vast amount of human-driven research still needed in this evolving field!