Research in the social sciences is facing a crisis of confidence due to the difficulty in reproducing and replicating research results. The MACRO-SCORE project will develop automated techniques for evaluating scientific claims and assessing the confidence of their reproducibility and replicability. Our approach centers on building a knowledge graph (KG) of scientific claims that integrates information across multiple levels of granularity, capturing fine-grained features from scientific articles using weakly-supervised information extraction and building a holistic view of research areas through network analysis of co-authorship and citation networks and social media. MACRO-SCORE will use explainable probabilistic models and develop new techniques to maintain robust performance avoiding gaming attacks. MACRO-SCORE promises to instill trust in social sciences research and transform the current approaches for training scientists and evaluating papers while dramatically advancing AI techniques for social network analysis, information extraction, gaming detection, and explainable AI.
This work was funded by the Defense Advanced Research Projects Agency with award W911NF-19-20271.