2_use_case_name,3_agency,3_abr,4_bureau,8_topic_area,8_question_type,10_commercial_ai,11_purpose_benefits,12_outputs,16_dev_stage,17_impact_type,18_date_initiated,19_date_acq_dev_began,20_date_implemented,21_date_retired,22_dev_method,23_contract_piids,24_hisp_support,25_hisp_name,26_public_service,27_public_info,28_iqa_compliance,29_contains_pii,30_saop_review,31_data_catalog,31_question_type,33_agency_data,34_data_docs,35_demo_features,35_question_type,37_custom_code,38_code_access,39_code_link,40_has_ato,41_system_name,42_dev_tools_wait,43_infra_provisioned,43_question_type,45_compute_request,45_question_type,47_timely_resources,47_question_type,49_existing_reuse,50_internal_review,51_extension_request,52_impact_assessment,53_real_world_testing,54_key_risks,55_independent_eval,56_monitor_postdeploy,57_autonomous_impact,57_question_type,59_ai_notice,59_question_type,61_adverse_impact,62_disparity_mitigation,63_stakeholder_consult,63_question_type,65_appeal_process,66_no_appeal_reason,67_opt_out,67_question_type Non-Intrusive Inspection (NII) 3D Imaging Tool,Department of Homeland Security,DHS,CBP,Law & Justice,,None of the above.,"Utilizes AI/ML to generate high resolution, rapid imaging of objects behind occlusions; create 3D images for existing processes without significant slowdowns; and provide a novel narcotics detection capability for the inspection of packages.",Detection alerts for Items of Interest.,Implementation and Assessment,Neither,7/1/2023,7/1/2023,,,Developed with contracting resources.,20139567,No,,,No,,No,Yes,No,,Images and data of baggage inspection.,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,No,No – agency does not have access to source code.,,No,,6-12 months,Yes,,No,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Babel,Department of Homeland Security,DHS,CBP,Law & Justice,,None of the above.,"CBP uses this tool to conduct targeted queries to aid CBP in open source research to monitor potential threats or dangers or identify travelers who may be subject to further inspection for violation of laws CBP is authorized to enforce or administer. ","Babel utilizes AI modules for Text detection and translation as well as object and image recognition to provide analysts with possible matches to manually review in a single interface versus doing multiple manual queries. The output is not singly used for action or decision making and are used to identify additional Open Source or Social Media of a person or identify additional selectors (such as phone and emails) that are previously unknown to CBP and compared by an analyst against Government systems to identify additional derogatory information. These factors can often eliminate the traveler from additional screening. ",Implementation and Assessment,"Rights-Impacting ",8/29/2023,8/29/2023,,,Developed with contracting resources.,70B03C23F00001059,No,,,No,,Yes,Yes,No,,"N/A - The Agency does not own any of the data used to train, fine-tune, and/or evaluate performance of the model(s) used in this use case.",,,,Yes,No – agency does not have access to source code.,,,,Less than 6 months,No,,,,,,,,Yes – Agency requested an extension for this use case.,Planned or in-progress.,,"Forthcoming. Office of Management and Budget (OMB) approved a compliance extension through November 30, 2025.  ",,,,,,,,,,,,,, Fivecast ONYX,Department of Homeland Security,DHS,CBP,Law & Justice,,None of the above.,"The system enhances CBP's capability to monitor, analyze, and assess threats related to border security by processing vast amounts of open-source data, CBP aims to detect potential risks, monitor emerging trends, and uncover connections between individuals, organizations, or networks involved in illegal activities such as human trafficking, smuggling, or terrorism. Thereby streamlining operations and bolster security measures while maintaining a proactive approach to emerging threats ","The system aggregates vast amounts of publicly and commercially available data from diverse sources, such as social media, news streams, company records, and even the dark web, to provide insights on potential threats and risks, that can be used as actionable intelligence reports, risk assessments, and alerts that support CBP personnel in real-time decision-making and long-term strategic planning. ",Implementation and Assessment,"Rights-Impacting ",8/29/2023,8/29/2023,,12/31/2024,Developed with contracting resources.,,No,,,No,,Yes,Yes,No,,"N/A - The Agency does not own any of the data used to train, fine-tune, and/or evaluate performance of the model(s) used in this use case.",,,,Yes,No – agency does not have access to source code.,,,,,,,,,,,,,Yes – Agency requested an extension for this use case.,Planned or in-progress.,,"Forthcoming. Office of Management and Budget (OMB) approved a compliance extension through November 30, 2025.  ",,,,,,,,,,,,,, Public Information Compilation for Travel Threat Analysis (Dataminr),Department of Homeland Security,DHS,CBP,Law & Justice,,None of the above.,"This tool significantly reduces the amount of time it takes for users to collect and compile commercially available open-source information when attempting to identify possible threats related to national security, border violence, CBP facilities, CBP employee safety and other topics with a CBP-nexus involving air, sea, and land travel to and/or from the U.S. ","The AI output is compiled publicly available information for awareness. CBP employees further research the information, including reading the source information, to determine if there is a possible threat. ",Acquisition and/or Development,Neither,9/15/2023,9/15/2023,,,Developed with contracting resources.,70B03C23F00001059,No,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Anomaly Detection in Non-Intrusive Inspection,Department of Homeland Security,DHS,CBP,Law & Justice,,None of the above.,"The model will identify irregularities or deviations from expected patterns that may indicate concealed contraband or threats while reducing overall review time. ","Specific areas within scanned images that show anomalies, possibly needing further screening. ",Acquisition and/or Development,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Advanced Analytics for X-ray Images (AAXI),Department of Homeland Security,DHS,CBP,Law & Justice,,None of the above.,"AAXI aims to address the problem of anomaly detection in empty commercial vehicles entering the United States at land border ports of entry. The AI models achieve this goal by encoding past X-Ray images of vehicular border crossings in a semantically meaningful way and comparing the current crossing to detect differences amongst the images to identify anomalies. Benefits include enhancement of the capability of humans to consistently detect items of interest/concern present (and possibly concealed) in vehicles crossing into the United States, and increased clearance rate at border crossings so that vehicles operating safely and lawfully may pass through the border faster.",AAXI produces bounding boxes around an anomaly or unidentifiable object(s) within an image or any portion of the image that cannot be identified or explained.,Acquisition and/or Development,"Rights-Impacting ",1/1/2022,6/1/2022,,,Developed with contracting resources.,IAA with S&T,No,,,,,,,No,,NII Images,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,,,,No,,,,,,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Automated Data Annotation,Department of Homeland Security,DHS,CBP,Mission-Enabling,,None of the above.,"The Automated Data Annotation system is used to manage training data, collaborate on internal annotation strategies, and to process vast amounts of raw data for AI/ML applications collected from several CBP missions.The intended purpose of the AI is used to generate domain specific training data to facilitate model training for specific mission use cases. The expected benefit would be increased accuracy and confidence in model development.","High-quality, labeled datasets in JSON format ready for machine learning. It also provides metadata, including annotation metrics and quality insights, to ensure accuracy and support model training workflows. ",Initiated,Neither,6/13/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, AI Enabled Autonomous Underwater Vehicle,Department of Homeland Security,DHS,CBP,Law & Justice,,None of the above.,"Customs and Border Patrol (CBP) Office of Field Operations (OFO) has identified significant potential for parasitic loads on vessels entering and exiting ports of entry. The current identification method is using dive teams or borrowing larger ROV units from Local, State, or Federal partners. Through autonomous systems, OFO can more efficiently and safely identify anomalies/items of interest. ","This AI helps a small submarine drone detect possible contraband or safety risks to prioritize additional checks by human divers. The technology allows for increased shared situational awareness in real time for OFO and strategic partners and improves mission planning and agent and officer safety, while reducing reactionary gaps. [this needs to be more clear about the ai system outputs - is the ai outputting suspected loads indicators? imagery? the closed loop fully autonomous navigation implies other self-looping outputs for navigation on obstacles and currents, are there any others?]",Initiated,Neither,1/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Relocatable Multi-Sensor System,Department of Homeland Security,DHS,CBP,Law & Justice,,None of the above.,"The system uses advanced sensor technology to differentiate valid items of interest (IOI), such as unmanned aircraft systems and humans, from other detections such as animals or other environmental objects. By integrating radar and other sensor data, the system filters out false alarms, ensuring more accurate identification of potential IOI. This capability enhances CBP's ability to focus on legitimate risks while minimizing the time spent on non-threatening activities, improving operational efficiency at border and security checkpoints. ","The outputs include real-time data identifying and categorizing potential items of interests, while filtering out false or non-relevant items of interest like animals. These outputs are used to provide situational awareness and support decision-making for CBP personnel. ",Initiated,Neither,5/24/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, AI/LLM to generate testable synthetic data,Department of Homeland Security,DHS,CBP,Mission-Enabling,,None of the above.,The purpose of using AI to generate test data for trade partners is to create more realistic data for trade partners to use for testing their systems before releasing new ACE capabilities. Currently there are many data issues where test data does not accurately reflect real production data which results in unrealistic failures when testing which result in wasted time and resources tracking down false positive errors during testing. By providing trade partners with more realistic test data the expectation is that testing times will be shorter and enhancements and capabilities can be delivered quicker.,The AI capability would generate test data without PII or other trade sensitive data and allow for more accurate simulation of production data.,Initiated,Neither,11/1/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Anomaly Detection COV Structure,Department of Homeland Security,DHS,CBP,Law & Justice,,None of the above.,Anomaly Detection COV Structure is an AI algorithm solution to assist CBP officers in adjudicating large-scale Non-Intrusive Inspection (NII) x-ray images. The solution will identify regions of interest within the structure of Commercially Owned Vehicles (COV) and detect anomalies within these regions. The algorithm will support the review and analysis conducted by image analysts during NII x-ray image adjudication ultimately reducing overall review time for NII images.,Bounding boxes around an anomaly or unidentifiable object(s) within an image or any portion of the image that cannot be identified or explained.,Initiated,Neither,1/26/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Anomaly Detection Homogenous Cargo,Department of Homeland Security,DHS,CBP,Law & Justice,,None of the above.,Anomaly Detection Homogenous Cargo is an AI algorithm solution to assist CBP officers in adjudicating large scale Non-intrusive Inspection (NII) x-ray images. The solution will identify anomalies within homogenous cargo contained in commercially owned vehicles (COVs). The algorithm will support the review and analysis conducted by CBP image analysts during NII x-ray image adjudication ultimately reducing overall review time for NII images.,Bounding boxes around an anomaly or unidentifiable object(s) within an image or any portion of the image that cannot be identified or explained.,Initiated,Neither,1/26/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Anomaly Detection POV Structure,Department of Homeland Security,DHS,CBP,Law & Justice,,None of the above.,Anomaly Detection POV Structure is an AI algorithm solution to assist CBP officers in adjudicating large-scale Non-Intrusive Inspection (NII) x-ray images. The solution will identify regions of interest within the structure of Privately Owned Vehicles (POV) and detect anomalies within these regions. The algorithm will support the review and analysis conducted by image analysts during NII x-ray image adjudication ultimately reducing overall review time for NII images while facilitating legitimate trade and travel.,Bounding boxes around an anomaly or unidentifiable object(s) within an image or any portion of the image that cannot be identified or explained.,Initiated,Neither,1/26/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, CBP Careers Bot - Leo,Department of Homeland Security,DHS,CBP,Government Services (includes Benefits and Service Delivery),,None of the above.,"Visitors to the U.S. Customs and Border Protection (CBP) careers website can engage with a decision-tree based chat bot to help access CBP career related information and drive users to take the next action such as contacting a recruiter, attending a career event, or applying for a CBP career to help access CBP career related information and drive users to take the next action. The addition of Natural Language Processing (NLP)-driven responses will allow for more natural, conversational interactions to increase usability and accuracy in provided information.",The AI-chatbot will provide natural language responses to user queries.,Initiated,Neither,1/1/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Computer Vision for Aerial Detection of Land and Open Water Items of Interest,Department of Homeland Security,DHS,CBP,Law & Justice,,None of the above.,"This capability will automate cross-border maritime threat detection, classification and tracking, enabling more efficient maritime border security surveillance. ",This capabilty will provide alerts and tracks of detected/classified maritime threats to system operators on a workstation user interface.,Initiated,Neither,1/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, AI for Software Delivery,Department of Homeland Security,DHS,CBP,Mission-Enabling,,None of the above.,The initial use case is to integrate AI into the development process to assist developers with code reviews so that when a request to modify code in a baseline is made by a developer AI is integrated into the continuous integration pipeline to examine the code for potential problems and inefficiencies. By reducing code review time and identifying potential issues earlier in the development CSPD expects to reduce the time to deliver changes and increase initial software quality.,"The AI model will identify coding errors and recommend fixes as well as make recommendations for improvement and optimization which the request reviewer will assess and determine what, if any, changes need to be made before the code is approved to go into the baseline.",Initiated,Neither,11/1/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Optical Counter - UAS Detection,Department of Homeland Security,DHS,CBP,Law & Justice,,None of the above.,"This capability will automate cross-border air & ground threat detection, classification and tracking, enabling more efficient border surveillance.",This capabilty will provide alerts and tracks of detected/classified air & ground threats to system operators on a workstation user interface.,Initiated,Neither,4/1/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Thermal Power Generation with Geoseismic IoI Detection and Classification,Department of Homeland Security,DHS,CBP,Law & Justice,,None of the above.,Utilize seismic sensor data to determine Item of Interest traffic in deployed locations. Increases situational awareness in austere environments and reduces need for battery replacement since the devices are self-charging.,Alert that an Item of Interest (IOI) is present and the confidence interval that the detected seismic activity is correctly classified.,Initiated,Neither,3/17/2021,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Underwater ROV,Department of Homeland Security,DHS,CBP,Law & Justice,,None of the above.,The expected benefit of the system is that users will be able to more quickly identify potential threats and contraband on maritime vessels. This identification will allow for the streamlined investigations of a ship's hull without the need for a dive team.,The AI system outputs provide an informed decision on if a potential threat or contraband is present on a maritime vessel hull.,Initiated,Neither,1/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Wellness and Physical Fitness Application ,Department of Homeland Security,DHS,CBP,Health & Medical,,None of the above.,"This is an internal AI powered personal workout coach for Customs and Border Patrol Staff. Wellness and Physical Fitness Application uses AI to provide increased user awareness of physical status and associated remedial actions, if necessary. Health and wellness awareness intends to mitigate human capital costs. ","Personalized physical fitness assessments and programs with real-time AI fitness monitoring and program adjustments. Metrics, graphs, and display for group and/or individual trends analyses. ",Initiated,Neither,10/1/2022,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, API Security Vulnerability Technology,Department of Homeland Security,DHS,CBP,Mission-Enabling,,None of the above.,"With the necessity of leveraging application programing interfaces (APIs) for applications across the enterprise, this AI technology is intended to run thousands of custom attack scenarios against APIs on a continuous basis. This will help to identify potential security vulnerabilities prior to production level deployments, and enable the enterprise to develop essential remediations, if required. Additionally, the techonlogy is intended to provide continuous monitoring on APIs to provide real-time alerts on new potential vulnerabilities. ","The AI system is intended to output custom reports on identified vulnerabilities within application programing interfaces (APIs). These reports will include the identified risk and a summary of the risk, as well as tailored remediation guidance based on the applications, environment, data, and tests conducted for end-users. ",Initiated,Neither,8/19/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Cyber Threat Detection,Department of Homeland Security,DHS,CBP,Mission-Enabling,,None of the above.,"The technology will provide premise and/or behind the firewall deployments to support all cloud environments. The technology will establish a decoy file sharing capability such as SharePoint, Shared Drive, etc. with tailored decoy products or reports and monitor and alert on the access and interactions with the decoys. ","The technology enables the accurate detection of malicious activity in the cyber environment for actionable insights so that leadership can identify threats and efficiently mitigate them while reducing the cost of other security defenses. With the technology, the user will be alerted of detected compromise, evaluation of the creation of decoys, and the monitoring/analysis of interaction with the decoys from the adversary. ",Initiated,Neither,3/29/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Multi-media Insight Tool,Department of Homeland Security,DHS,CBP,Mission-Enabling,,None of the above.,"The system enhances CBP's ability to monitor and analyze surveillance footage from existing USBP camera technology, enabling real-time detection of anomalies, and tracking of Items of interest within the video frame. It improves situational awareness, streamlines the identification process, and supports faster, more accurate decision-making, ultimately enhancing security and operational efficiency.","Outputs include real-time alerts on activities that should be watched, visual annotation to identify items of interest. ",Initiated,Neither,3/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Position Description Generation and Evaluation,Department of Homeland Security,DHS,CBP,Mission-Enabling,,None of the above.,"Faster and more accurate creation of PDs, reducing admin burden on field and HQs as well as HR specialists, allowing agency to get more done with less staff.","Creation of accurate position descriptions, verified by Human Resources Specialists",Initiated,Neither,11/8/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Source Code Development Tool,Department of Homeland Security,DHS,CBP,Mission-Enabling,,None of the above.,"This capability will enable end users to develop software faster and more efficiently through the use of a generative artificial intelligence coding assistant. ","This capability will provide functional software code for end users. ",Initiated,Neither,11/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, AI Curated Synthetic Data,Department of Homeland Security,DHS,CBP,Mission-Enabling,,None of the above.,The availability of rare event/outlier data or labeled data that can be used to train and automate detection is severely limited. The technology enhances available libraries of true positives and normal scans from CBP Non-Intrusive Inspection (NII) systems and supports development of Automated Threat Recognition (ATR) algorithms designed to quickly identify items of interest.,"Through the technology the end user will have access to enhance existing, deployed algorithms to increase detection performance.",Retired,,10/1/2022,,,10/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Integrated Defense and Security Solutions (IDSS),Department of Homeland Security,DHS,CBP,Law & Justice,,None of the above.,"The systems improves the screening efficiency and accuracy of contraband detection in international express consignment and mail inspection. ","The system provides a segmented image, highlighting anamolies for further inspection by CBP personnel.",Initiated,Neither,1/1/2019,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Advance RPM Maintenance Operating Reporter (ARMOR),Department of Homeland Security,DHS,CBP,Mission-Enabling,,None of the above.,"ARMOR will shorten time to service/repair/maintenance of radiation portal monitors by two weeks. ARMOR will allow better distribution of resources (travel, spare parts, etc.) and expected cost decrease could be 25-50%. Through decreased outage time, and prediction of equipment degradation, ARMOR will increase radiological/nuclear (R/N) security on US borders.",The system will provide a listing of malfunctioning RPMs categorized by issue severity and predicted date of failure. The outputs will be used to create service tickets.,Acquisition and/or Development,Safety-Impacting,10/1/2021,10/1/2021,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, RAPTOR (Rapid Tactical Operations Reconnaissance),Department of Homeland Security,DHS,CBP,Law & Justice,,None of the above.,"RAPTOR will significantly increase domain awareness and the agency’s ability to engage in intelligence-driven operations. The AI capability acts as a force multiplier and saves personnel from analyzing video feed from a stationary camera and manually noting all boat identifiers, improving their ability to respond quickly to potential threats and gather critical intelligence for law enforcement and border control operations.",Text transcription of vessel registration/documentation data and photographs of the vessel.,Initiated,Neither,4/1/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Vessel Detection,Department of Homeland Security,DHS,CBP,Law & Justice,,None of the above.,"Current surveillance technology does not have machine assisted classification of targets on screen, making it harder for human operators to decipher legitimate from illegitimate traffic in times of high volumes of legitimate traffic. Project intends to support human operator identification and classification of potential illicit vessels. Benefits would be more efficient detection and resolution of IOIs, especially during times of high-volume traffic. ",Alerts and tracks of detected Items of Interest (IOI)s to human operator workstations. ,Initiated,Neither,10/1/2021,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, AI for Autonomous Situational Awareness,Department of Homeland Security,DHS,CBP,Law & Justice,,None of the above.,"The AI for autonomous situational awareness system is intended to use IoT sensor kits to covertly detect and track illicit cross-border traffic in remote locations. The system will leverage a motion image/video system enhanced with Artificial Intelligence that is capable of vehicle detection and direction determination, to allow for more active monitoring of remote camera feeds that prompt operators to review the items of interest once detected. This will reduce the overall operator burden of camera monitoring and allow users to accomplish other tasks or system management. ","Detections of items of interest being displayed with bounding boxes around the IOI in the image frame and an initial determination of ""human, animal, vehicle""""",Initiated,Neither,10/1/2020,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, RVSS Legacy Overhauled System Project,Department of Homeland Security,DHS,CBP,,,None of the above.,"Video Computer Aided Detection (VCAD) (also known as Matroid AI) is software that enables CBP end users to create and share vision detectors. VCAD detectors are trained computer vision models that recognize objects, people, and events in any image or video stream. Once a detector is trained, it can monitor streaming video in real time, or efficiently search through pre-recorded video data or images to identify objects, people, and events of interest. Users can view detection information via a variety of reports and alert notifications to process and identify important events and trends. Detection data is also available through VCAD's powerful developer Application Programming Interface (API) and language specific clients, so CBP applications can be integrated with the power of computer vision.","This use case is inactive. Please see the ""intended purpose and expected benefits"" for a full description of this use case. ",Retired,,10/1/2019,10/1/2020,6/1/2023,N/A -Consolidated into another use case.,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Agent Portable Surveillance,Department of Homeland Security,DHS,CBP,,,None of the above.,"The agent portable surveillance system is a backpack mobile unit meant for single agent deployments. The system identifies border activities of interest by using AI/ML to analyze data from Electro-Optical/Infra-Red cameras and radar. When an activity is detected, the system sends the information to agents through the Team Awareness Kit (TAK). Detections are shared with CBP TAK users to enhance efficiency and agent/officer safety.","This use case is inactive. Please see the ""intended purpose and expected benefits"" for a full description of this use case. ",Retired,,,,,10/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Open-Source News Aggregation,Department of Homeland Security,DHS,CBP,,,None of the above.,"The platform enables users to make better decisions faster by identifying and forecasting emerging events on a global scale to mitigate risk, recognize threats, greatly enhance indications and warnings, and provide predictive intelligence capabilities. The AI/ML models enable rapid access to automated intelligence assessments by fusing, processing, exploiting and analyzing open sources of data (including news, social media, economic indicators, governance indicators, travel warnings, weather and other sources). This system is an immediate and substantial force multiplier that shifts the traditional approach of monitoring and assessing the operational environment to focus on the forecast of the future geopolitical, socio, and economic environment.","This use case is inactive. Please see the ""intended purpose and expected benefits"" for a full description of this use case. ",Retired,,,,,10/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Autonomous Aerostat,Department of Homeland Security,DHS,CBP,,,None of the above.,"Aerostat capability that uses three tethers instead of the traditional single tether, coupled with advanced weather sensors, analytic capabilities, and powerful winches. The AI/ML model is used to detect the need to launch and land based on weather. It also leverages AI and robotics to autonomously launch and recover the aerostat during inclement weather events without the need for on-site staffing, allowing the aerostat to operate autonomously, saving time and manpower. ","This use case is inactive. Please see the ""intended purpose and expected benefits"" for a full description of this use case. ",Retired,,10/1/2019,10/1/2018,,10/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Geospatial Imagery Utilizing Annotation,Department of Homeland Security,DHS,CBP,,,None of the above.,"Leverages a commercial constellation of Synthetic Aperture Radar (SAR) satellites with readily available data, capable of imaging any location on Earth, day, and night, regardless of cloud cover. Utilizes AI, including machine vision, object, detection, object recognition, and annotation to detect airframes, military vehicles, and marine vessels, as well as built-in change detection capabilities for disaster response missions.","This use case is inactive. Please see the ""intended purpose and expected benefits"" for a full description of this use case. ",Retired,,10/1/2021,1/1/2022,,10/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Use of technology to identify proof of life,Department of Homeland Security,DHS,CBP,,,None of the above.,Mobile applications rely on product for liveness detection to avoid use of spoofed or fraudulent images by bad actors. Being able to accept submitted data with confidence that the submitting individual is who and where they claim to be is critical to the functionality of the app within the agency environment.,"This use case is inactive. Please see the ""intended purpose and expected benefits"" for a full description of this use case. ",Retired,,10/1/2020,10/1/2020,10/1/2023,N/A -Consolidated into another use case.,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Integrated Digital Environment,Department of Homeland Security,DHS,CBP,,,None of the above.,"The Integrated Digital Environment provides managers with a better understanding of end user workflows, most and least used applications, and opportunities for improvement. The AI/ML model applies to end user activity data (e.g., use of applications, flow between applications) to help CBP identify opportunities for more efficient or effective configuration of interfaces, use of resources, or development and deployment of CBP’s applications. It tailors analytics and insight generation to allow metrics gathering, usage recording/observation, dashboarding, and workflow experimentations/suggestions to support analysts utilizing the entire suite of agency and open-source data systems. It also customizes existing capabilities to allow the exact automations needed for agency applications and systems, creating an integrated digital environment for greater connectivity and security between applications, and better ability for CBP administrators to manage and optimize use of applications by end users. ","This use case is inactive. Please see the ""intended purpose and expected benefits"" for a full description of this use case. ",Retired,,10/1/2021,4/1/2022,,10/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Data and Entity Resolution,Department of Homeland Security,DHS,CBP,Mission-Enabling,,None of the above.,Entity resolution for ES multiple independent valid requirements that may be similar or able to be consolidated.,"""Uses machine learning and human input to leverage all data attributes to resolve entities, rather than traditional brute force rules Automates process of unifying data sources and resolving entities at enterprise speed""",Retired,,10/1/2021,4/1/2022,10/1/2019,11/30/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Port of Entry Risk Assessments,Department of Homeland Security,DHS,CBP,,,None of the above.,"CBP utilizes AI to develop, inform, and augment risk assessment processes that evaluate trade and travel data in real-time.  AI methods are applied to CBP data holdings, and the results are used to inform decision making.  These tools are continuously evaluated to ensure accuracy and precision, and support CBP’s core mission as part of the layered risk assessment strategy. ","This use case is inactive. Please see the ""intended purpose and expected benefits"" for a full description of this use case. ",Retired,,,,,10/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Traveler Verification Service (TVS),Department of Homeland Security,DHS,CBP,,,None of the above.,"The Traveler Verification Service (TVS) provides CBP a biometric entry/exit system to record arrivals and departures to and from the United States. CBP uses TVS as its backend matching service for all biometric entry and exit operations that use Facial Comparison. CBP creates localized photo ""galleries"" from images captured during previous entry inspections, photographs from U.S. passports and U.S. visas, and photographs from other DHS encounters. The images are converted into templates and the actual photograph is discarded. The templates are securely stored, and are what make up the TVS gallery. The templates are then used by the Facial Comparison system to verify a traveler's identity when they arrive or depart the U.S. When the traveler presents him or herself for entry, or for exit, the traveler will encounter a camera connected to TVS. This camera matches live images with the existing photo templates from passenger travel documents. Once the camera captures a quality image and the system successfully matches it with historical photo templates of all travelers from the gallery associated with that particular manifest, the traveler proceeds to inspection for admissibility by a CBP Officer, or exits the United States.","This use case is inactive. Please see the ""intended purpose and expected benefits"" for a full description of this use case. ",Retired,,,,,N/A - Deconsolidation of several use cases,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, I4 Viewer Matroid Image Analysis,Department of Homeland Security,DHS,CBP,,,None of the above.,"Matroid is a software that enables CBP end users to create and share vision detectors. Matroid detectors are trained computer vision models that recognize objects, people, and events in any image and in video streams. Once a detector is trained, it can monitor streaming video in real time, or efficiently search through pre-recorded video data or images to identify objects, people, and events of interest. Users can view detection information via a variety of reports and alert notifications to process and identify important events and trends. Detection data is also available through Matroid’s powerful developer Application Programming Interface and language-specific clients, so CBP applications can be integrated with the power of computer vision. ","This use case is inactive. Please see the ""intended purpose and expected benefits"" for a full description of this use case. ",Retired,,,,,10/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Autonomous Maritime Awareness,Department of Homeland Security,DHS,CBP,Law & Justice,,None of the above.,"The technology is equipped to operate in low temperatures, ice, and snow enable real-time surveillance for continuous monitoring of maritime activity, even in harsh conditions, while providing enhanced detection capabilities to identify threats such as unauthorized vessels or smuggling activities in difficult-to-navigate regions. ","The system provides real-time notifications of suspicious activities, continuous tracking data on vessel locations and movements, and detailed threat assessments to support enforcement actions in cold-weather environments. ",Retired,,10/1/2020,10/1/2020,1/30/2024,6/21/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Advanced Trade Analytics Platform (ATAP),Department of Homeland Security,DHS,CBP,Diplomacy & Trade,,None of the above.,"To create efficiencies and unlock key insights in CBP's trade mission execution through the application of data analytics, machine learning, and AI.",Model output is provided in dashboards and other visualization mechansims for operator assessment and action determination.,Operation and Maintenance,Neither,7/1/2019,12/1/2019,3/7/2022,,Developed with contracting resources.,BPA: 70B06C20A00000036 Call Order 1: 70B06C20F00000895 Call Order 2: 70B06C21F00000842 Call Order 3: 70B06C22F00000700 Call Order 4: 70B06C23F00000819 Call Order 5: 70B06C24F00000450,No,,,No,,Yes,Yes,Yes,,"ATAP relies on CBP source system information from CBP's ACE, ATS, and SEACATS systems, including import/export filing information, compliance reviews, targeting, seizure, and fine/penalty information.","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,ATAP,6-12 months,Yes,,Yes,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, CBP Employee Experience,Department of Homeland Security,DHS,CBP,Mission-Enabling,,None of the above.,"CBP Employee Experience is intended to ingest, interpret, and operationalize employee experience data originating from survey results and operational data to deliver real time insights related to the experience of USBP recruits, applicants, and employees. These metrics inform HRM leadership of opportunities for process improvement in order to meet congressionally mandated hiring targets and retain a qualified workforce.","Real time insights related to the experience of USBP recruits, applicants, and employees.",Operation and Maintenance,Neither,12/1/2023,5/1/2022,10/1/2023,,Developed with contracting resources.,024-000005282,No,,,No,,No,Yes,Yes,,User input is captured in feedback surveys.,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,No,No – agency does not have access to source code.,,Yes,MASS,6-12 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, Passive Body Scanner,Department of Homeland Security,DHS,CBP,Law & Justice,,None of the above.,"PBS is intended to enhance situational awareness in pedestrian traveler processing to aid CBP officers in observing potentially dangerous objects or contraband in a timely manner and pursuant to CBP’s border search authority. ","This algorithm highlights areas on a person where potential objects may be blocking the subject's expected body heat and displays these areas on live video image, monitored by a CBP officer. The highlighted areas may show the locations of carried objects, which could be potential weapons or contraband. ",Operation and Maintenance,"Rights-Impacting ",8/18/2022,9/29/2022,9/29/2023,,Developed with contracting resources.,70B03C22A00000031,No,,,No,,No,Yes,No,,"N/A - The Agency does not own any of the data used to train, fine-tune, and/or evaluate performance of the model(s) used in this use case.","Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,Yes,No – agency does not have access to source code.,,Yes,Passive Body Scanner,6-12 months,No,,No,,Other,These are stand-alone AI models that are provided as part of the equipment. They do not touch IT infrastructure. The systems are air gapped.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,Planned or in-progress.,,"Forthcoming. Office of Management and Budget (OMB) approved a compliance extension through November 30, 2025.  ",,,,,,,,,,,,,, Unmanned Aircraft Collision Avoidance (Skydio),Department of Homeland Security,DHS,CBP,Law & Justice,,None of the above.,The platform operates on video feed only which in turn activates the obstacle avoidance on the aircraft where the AI capabilities are housed.  The system supports the streamlined intake process while maintaining the accuracy and reliability of identity verification.,"The pilot of the Skydio X2D will receive a visual alert on the hand controller, indicating a possible collision.",Operation and Maintenance,Neither,6/1/2022,6/1/2022,10/1/2022,,Developed in-house.,,No,,,No,,No,No,No,,Still capture pictures and video files.,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,No,No – agency does not have access to source code.,,Yes,Skydio X2D,Less than 6 months,No,,No,,Other,N/A,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, CBP Translate,Department of Homeland Security,DHS,CBP,Law & Justice,,None of the above.,"CBP Translate is used to facilitate clear communication between officers and non-English-speaking individuals during border interactions. The AI enhances efficiency by reducing language barriers, supports diverse languages for greater inclusivity, and ensures accurate communication to minimize misunderstandings during critical procedures. ","The system provides real-time translations of spoken or written communication, detects individuals' languages for accurate translation, and generates logs of interactions for review, streamlining processes and ensuring clear communication. This output is used to support necessary interpretation but sworn statements and official communications for law-enforcement purposes are moved to stardard translation services when necessary for formal documenation and questioning. ",Operation and Maintenance,"Rights-Impacting ",,,,,,,,,,Yes,"In progress, working as part compliance extension.",Yes,Yes,,,"N/A - The Agency does not own any of the data used to train, fine-tune, and/or evaluate performance of the model(s) used in this use case.","Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,CBP Translate,Less than 6 months,,,Other,Working as part of compliance extension.,,,,,Yes – Agency requested an extension for this use case.,Planned or in-progress.,,"Forthcoming. Office of Management and Budget (OMB) approved a compliance extension through November 30, 2025.  ",,,,,,,,,,,,,, Entity Resolution,Department of Homeland Security,DHS,CBP,Law & Justice,,None of the above.,Detection research of force labor within supply chain utilizing analytical AI platform.,Detection of potential force labor within supply chain.,Operation and Maintenance,Neither,10/1/2021,10/1/2021,5/1/2023,,Developed with contracting resources.,2014-14031000011,No,,,No,,No,Yes,No,,"N/A - The Agency does not own any of the data used to train, fine-tune, and/or evaluate performance of the model(s) used in this use case.","Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,No,No – agency does not have access to source code.,,No,,6-12 months,Yes,,No,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Supervised Traveler Identity Verification Services (Officer Initiated),Department of Homeland Security,DHS,CBP,Law & Justice,,None of the above.,"The TVS Biometric matching service is a cloud-based facial biometric matching service that enables CBP to match a passenger’s identity against a trusted source, throughout the travel continuum which improves traveler facilitation and reduces manual identity verification",Leverages DHS facial matching technologies to provide a match or no match response,Operation and Maintenance,"Rights-Impacting ",4/1/2013,4/1/2016,9/1/2017,,Developed with contracting resources.,024-000005265,Yes,Customs and Border Protection, Enabling trusted travelers to enter the country quickly with Global Entry (GE),No,,Yes,Yes,Yes,,Border Crossing Information,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Traveler Verification Service,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,Yes,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.",This AI system has been tested in operational or real-world environments and risks have been identified and mitigated. Information about risks and mitigations for this use case may be law enforcement sensitive; DHS continues to review details for potential future disclosure in accordance with applicable law and policy.,Yes – by the CAIO,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,"Online - in the terms or instructions for the service.,In-person,Other",Facial matching and privacy notices are posted where facial recognition technology is in use.,No,The threshold for the biometric matching was tested extensively with a variety of face types for several months to establish a match threshold for the identification.,"Direct user testing,General solicitations of comments from the public",,Yes,,Yes, Semi-Supervised Traveler Identity Verification Services (Traveler Initiated),Department of Homeland Security,DHS,CBP,Law & Justice,,None of the above.,"The TVS Biometric matching service is a cloud-based facial biometric matching service that enables CBP to match a passenger’s identity against a trusted source, throughout the travel continuum which improves traveler facilitation and reduces manual identity verification.",Leverages DHS facial matching technologies to provide a match or no match response,Operation and Maintenance,"Rights-Impacting ",6/1/2017,7/1/2023,2/1/2024,,Developed with contracting resources.,024-000005265,No,,,No,,Yes,Yes,Yes,,Trusted Traveler Information,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Traveler Verification Service,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,Yes,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.",This AI system has been tested in operational or real-world environments and risks have been identified and mitigated. Information about risks and mitigations for this use case may be law enforcement sensitive; DHS continues to review details for potential future disclosure in accordance with applicable law and policy.,Yes – by the CAIO,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,"Online - in the terms or instructions for the service.,In-person,Other",Facial matching and privacy notices are posted where facial recognition technology is in use.,No,The threshold for the biometric matching was tested extensively with a variety of face types for several months to establish a match threshold for the identification.,"Direct user testing,General solicitations of comments from the public",,Yes,,Yes, 3rd Party Traveler Identity Verification Services,Department of Homeland Security,DHS,CBP,Law & Justice,,None of the above.,"The TVS Biometric Air Exit solution is a cloud-based facial biometric matching service that enables CBP, External Partners, and Other Government Agencies (OGA) to match a passenger’s identity against a trusted source, throughout the travel continuum which improves traveler facilitation and reduces manual identity verification. ","Leverages DHS facial matching technologies to provide a match or no match response. ",Operation and Maintenance,"Rights-Impacting ",4/1/2013,12/1/2015,5/1/2017,,Developed with contracting resources.,024-000005265,No,,,No,,Yes,Yes,Yes,,Border Crossing Information.,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Traveler Verification Service,Less than 6 months,No,,No,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,Yes,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.",This AI system has been tested in operational or real-world environments and risks have been identified and mitigated. Information about risks and mitigations for this use case may be law enforcement sensitive; DHS continues to review details for potential future disclosure in accordance with applicable law and policy.,Yes – by the CAIO,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,"Online - in the terms or instructions for the service.,In-person,Other",Facial matching and privacy notices are posted where facial recognition technology is in use.,No,The threshold for the biometric matching was tested extensively with a variety of face types for several months to establish a match threshold for the identification.,"Direct user testing,General solicitations of comments from the public",,Yes,,Yes, Mobile Traveler Identity Verification,Department of Homeland Security,DHS,CBP,Law & Justice,,None of the above.,"The TVS Biometric matching service is a cloud-based facial biometric matching service that enables CBP to match a passenger’s identity against a trusted source, throughout the travel continuum which improves traveler facilitation and reduces manual identity verification",Provides a match or no match response,Operation and Maintenance,Neither,6/1/2017,9/1/2017,1/1/2022,,Developed with contracting resources.,024-000005265,Yes,Customs and Border Protection, Enabling trusted travelers to enter the country quickly with Global Entry (GE),No,,Yes,Yes,Yes,,Border Crossing Information/ Trusted Traveler Information,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Traveler Verification Service,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, Traveler Identity Verification Services (Vetting),Department of Homeland Security,DHS,CBP,Law & Justice,,None of the above.,CBP's Traveler Identity Verification Services (Vetting) utilizes facial recognition technology to enhance threat identification by matching travelers' biometrics against records of concern. ,"When the system identifies a potential match to concerning records, CBP personnel conducts a manual facial comparison to determine whether the record is likely associated with the individual.",Operation and Maintenance,"Rights-Impacting ",12/1/2017,1/1/2018,12/1/2018,,Developed with contracting resources.,024-000005265,No,,,No,,Yes,Yes,Yes,,Border Crossing Information,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Traveler Verification Service,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,Yes,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.",This AI system has been tested in operational or real-world environments and risks have been identified and mitigated. Information about risks and mitigations for this use case may be law enforcement sensitive; DHS continues to review details for potential future disclosure in accordance with applicable law and policy.,Yes – by the CAIO,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,"In-person,Other",Facial Matching and privacy notices where FR tech is in use.,No,The threshold for the biometric matching was tested extensively with a variety of face types for several months to establish a match threshold for the identification.,None of the above,,Yes,,"No – This AI use case is not subject to the opt-out requirement because the AI functionality is solely used for the prevention, detection, and investigation of fraud or cybersecurity incidents, or the conduct of a criminal investigation.", Customs Broker License Exam - Proctor Support,Department of Homeland Security,DHS,CBP,Education & Workforce,,None of the above.,"The model supports remote proctoring of the exam and ensure the integrity of the testing process, by ensuring the exam is conducted under secure conditions, preventing cheating or fraud, while also verifying the identity of exam takers to confirm they meet the necessary requirements. ","Integrity reports, identity confirmation, proctoring complicance feedback, and test results. ",Operation and Maintenance,Neither,10/1/2022,10/2/2022,3/31/2023,,Developed with contracting resources.,N/A - Aligned with the Proctoring contract.,No,,,No,,Yes,Yes,No,,"N/A - The Agency does not own any of the data used to train, fine-tune, and/or evaluate performance of the model(s) used in this use case.","Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,No,No – agency does not have access to source code.,,No,,Less than 6 months,No,,No,,Other,N/A,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, ERNIE,Department of Homeland Security,DHS,CBP,Law & Justice,,None of the above.,"The model enhances threat detection and prioritizes high-risk targets, improving operational efficiency and national security.",The model provides real-time risk assessments and alerts for potential threats detected by the Radiation Portal Monitors. It also provides prioritized recommendations for further screening based on the analysis of radiation data.,Operation and Maintenance,Both,10/1/2017,10/1/2017,,,Developed with contracting resources.,Developed under CWMD contract.,No,,,No,,No,Yes,Yes,,Numerical data from RPM radiation detectors and ERNIE assessments.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Automated Radiological Data Integration System - Cloud (ARDIS-C),More than 12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,Yes,,This AI system has been tested in operational or real-world environments and risks have been identified and mitigated. Information about risks and mitigations for this use case may be law enforcement sensitive; DHS continues to review details for potential future disclosure in accordance with applicable law and policy.,Yes – by the CAIO,,,,,,,,,,,,, Autonomous Surveillance Tower (AST),Department of Homeland Security,DHS,CBP,Law & Justice,,None of the above.,"AST machine learning assisted system is augmenting the U.S. Border Patrol by enhancing the capabiltiies of individual users when carrying out the domain awareness mission. The expected benefit is to have ability of single person to monitor magnitude greater area than could be done with conventional CCTV or human surveillance. The ultimate outcome for the agency and the public is greater availability of the agents to solve and address more complex tasks and allow for better strategic/tactical deployment of existing resources and personnel. ","The AI provides alerts when it detects the presence of an IoI (i.e., persons, vehicles, animals) in the image frame. With regard to persons, this computer vision application is trained to determine if the object in the image frame is a person with a certain of level of confidence and not another object that may be shaped similarly to a person. After the alert of a detection, a trained agent or user, reviews the image to identify and classify the activity taking place. The AI merely alerts to the presence of an item it was trained to detect. This is not a biometric system and does not identify or track specific individuals. ",Operation and Maintenance,Neither,10/1/2018,10/1/2019,1/1/2020,,Developed with both contracting and in-house resources.,70B02C20D00000019,No,,,No,,No,No,Other,Program created its own new data through new inputs either in real world or in real world simulated scenarios,Meta-data and data created by CBP. Generally comprised of agent adjudications of autonomous sensory inputs of items of interests by the system.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,Yes,No – agency does not have access to source code.,,Yes,Autonomous Surveillance Towers,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, Automated Item of Interest Detection,Department of Homeland Security,DHS,CBP,Law & Justice,,None of the above.,"The software analyzes images and video that are taken by operationally deployed equipment, which are then fed into CBP systems for review by USBP agents and personnel. It provides quick identification of people either crossing into the U.S. at a time and place other than designated for entry or those already inside the U.S. trying to elude capture, as well as the ability for human operators to quickly determine if subjects in an image are, in fact, human.",The system creates a layer which is overlaid over the image to produce a box around items of interest it has determined to be likely human beings.,Operation and Maintenance,Neither,,,,,Developed with contracting resources.,70B06C24F00000475,No,,,No,,No,Yes,No,,All of the image data fed to the models are owned by USBP.,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,Yes,No – agency does not have access to source code.,,Yes,I4 Viewer,6-12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, CBP One,Department of Homeland Security,DHS,CBP,Law & Justice,,None of the above.,"TVS uses facial recognition to compare live or uploaded images with CBP's database, enabling real-time identity verification. This automation streamlines border processes, enhances accuracy, and reduces fraud.","The system outputs include identity match confirmation, fraud alerts, and traveler status updates for clearance in processes like boarding or border crossing. ",Operation and Maintenance,Both,4/6/2023,4/6/2023,5/11/2023,,Developed with contracting resources.,024-000005265,No,,,No,,Yes,Yes,Yes,,CBP One submission information.,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Traveler Verification Service,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,Yes,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.",This AI system has been tested in operational or real-world environments and risks have been identified and mitigated. Information about risks and mitigations for this use case may be law enforcement sensitive; DHS continues to review details for potential future disclosure in accordance with applicable law and policy.,Yes – by the CAIO,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,Online - in the terms or instructions for the service.,,No,The threshold for the biometric matching was tested extensively with a variety of face types for several months to establish a match threshold for the identification.,Direct user testing,,Yes,,Yes, Unified Processing/Mobile Intake,Department of Homeland Security,DHS,CBP,Law & Justice,,None of the above.,The purpose is to facilitate the biometric identification of individuals as they are encountered by CBP for the purpose of expedited processing.,Leverages CBP facial matching technologies to provide a match or no match response,Operation and Maintenance,"Rights-Impacting ",12/1/2017,1/1/2018,3/1/2022,,Developed with contracting resources.,024-000005265,No,,,No,,Yes,Yes,Yes,,Border Crossing Information,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Traveler Verification Service,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,Yes,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.",This AI system has been tested in operational or real-world environments and risks have been identified and mitigated. Information about risks and mitigations for this use case may be law enforcement sensitive; DHS continues to review details for potential future disclosure in accordance with applicable law and policy.,Yes – by the CAIO,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,"Online - in the terms or instructions for the service.,In-person,Other",Facial matching and privacy notices are posted where facial recognition technology is in use.,No,The threshold for the biometric matching was tested extensively with a variety of face types for several months to establish a match threshold for the identification.,Direct user testing,,Yes,,"No – This AI use case is not subject to the opt-out requirement because the AI functionality is solely used for the prevention, detection, and investigation of fraud or cybersecurity incidents, or the conduct of a criminal investigation.", Cyber Threat Analysis (Recorded Future),Department of Homeland Security,DHS,CBP,Law & Justice,,None of the above.,"Cyber Threat Analysis quickly populates query results when searching against adversary tactics, techniques, and procedures, establishing a threat scorecard. This service can also provide cyber risk scorecards for third party vendors, companies, and organizations.",Actionable intelligence supporting Security Operations Center (SOC) and Cyber Risk Management (CRM) investigations and reports.,Operation and Maintenance,Neither,4/1/2020,4/1/2020,,,,,No,,,No,,No,,No,,"N/A - The Agency does not own any of the data used to train, fine-tune, and/or evaluate performance of the model(s) used in this use case.",,,,No,No – agency does not have access to source code.,,No,,,No,,No,,,,,,,,,,,,,,,,,,,,,,, Vault Access Log (SPVAA),Department of Homeland Security,DHS,CBP,Mission-Enabling,,None of the above.,"The system will enhance monitoring and minimizes the risk of unauthorized access, contributing to stronger security protocols for handling seized property.",Leverages DHS facial matching technologies to provide a match or no match response.,Operation and Maintenance,Neither,12/1/2020,1/1/2021,8/1/2022,,Developed with contracting resources.,024-000005265,No,,,No,,Yes,Yes,No,,Personal Identity Verification (PIV) card,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Traveler Verification Service,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, HTS Classifier,Department of Homeland Security,DHS,CBP,Law & Justice,,None of the above.,"CBP’s HTS Classifier improves trade compliance and enhances cargo risk assessment by streamlining the classification of goods, enabling better integration with machine learning systems, and refining entity risk evaluations. It identifies potential threats linked to specific cargo types and prior violations by categorizing goods based on their descriptions and attributes. These improvements contribute to faster, more accurate classification and risk-based targeting, which strengthens security and facilitates trade.","The HTS Classifier produce outputs that map cargo commodity descriptions to their most probable tariff codes, enhancing classification accuracy. These outputs integrate seamlessly into broader threat-specific risk models, providing features to support predictive risk assessments in cargo security.",Operation and Maintenance,Neither,10/1/2020,10/1/2020,10/1/2020,,Developed with contracting resources.,,No,,,No,,Yes,Yes,Yes,,"This model leverages data provided by carriers within the Automated Commercial Environment (ACE), as well as transformations of that data within the Automated Targeting System (ATS).","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Automated Targeting System (ATS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, CISAChat,Department of Homeland Security,DHS,CISA,Mission-Enabling,,None of the above.,"Currently, multiple CISA program offices are using contractor staff to review pre-production content and other internal materials to develop summaries, key themes, and improve clarity. Leveraging a Generative AI solution improves internal agency Customer Experience (CX) and saves staff time. ",LLM generated response to the questions posed on uploaded content. ,Implementation and Assessment,Neither,12/13/2023,1/1/2024,,,Developed with both contracting and in-house resources.,CEEOSS-Task Order No.- 70RCSJ23FR0000001 DSS-70RCSJ24FC0000018 - IT O&M,No,,,No,,No,Yes,Other,"CISA maintains an agency-wide data asset inventory, but it was not needed for this use case as CISAChat only currently operates on content that will be publicly posted.",Current data used is pre-publication content that has already been approved.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Critical Infrastructure Network Anomaly Detection,Department of Homeland Security,DHS,CISA,Mission-Enabling,,None of the above.,"This use case delivers improved internal government tools for hunting and detection of malicious threat actors on critical infrastructure networks. It automates manual data fusion and correlation processes and highlights potential anomalies, allowing CISA analysts to focus more time on hunting adversaries. ","An interface is provided for analysts to query cybersecurity data, and dashboards are provided with potential cybersecurity alerts, including anomalies detected through predictive models and rule-based heuristics.",Initiated,Neither,10/1/2022,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Malware Reverse Engineering,Department of Homeland Security,DHS,CISA,Mission-Enabling,,None of the above.,"This use case delivers improved internal government tools for reverse engineering of malware and speeding the development of cyber threat intelligence that can be shared across the government and with CISA partners. Threat actors can leverage the same malware for long periods of time, so having the ability to improve analysis and generation of shareable cyber threat intelligence forces threat actors to spend more resources generating new malware. Machine learning and other analytical tools are leveraged to guide malware analysts and automate elements of the manual reverse engineering process. Automation of tasks such as triage and indicator extraction allow threat hunting analysts to meet high demand and focus more on adversary response. ","A report is generated from malware samples submitted to the analysis pipeline that is then used by human analysts to facilitate the malware triage process. Additional recommendations are displayed via plugins to reverse engineering tools. ",Initiated,Neither,10/1/2022,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Draft Tailored Summaries of Media Materials for Different Publication Channels,Department of Homeland Security,DHS,CISA,Mission-Enabling,,None of the above.,The Large Language Models (LLMs) will summarize information from historical publications and pre-production documents intended for external publication.,"An interface is provided for pre-publication documents to be uploaded. The system will then automatically generate appropriate messaging using approved templates and tag the documents for review prior to publication. ",Initiated,Neither,11/1/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Security Information and Event Management (SIEM) Alerting Models,Department of Homeland Security,DHS,CISA,,,None of the above.,"CISA Threat hunting and Security Operations Center (SOC) analysts are provided with terabytes of log data per day. Manually developed detection alerts and automatic correlation in Security Information and Event Management (SIEM) tools are common, but not comprehensive. Many cyber attacks can be probabilistically determined given sufficient training data and time. Analysts use automated tooling to further refine the alerts they receive and produce additional automated alerts based on aggregated information and curated subject matter expertise. This can be further supported by the development of AI-assisted agents. Analysts can interact with such agents in a conversational manner to produce the input needed for complex search queries for behaviors or patterns in traffic that the agent then rapidly creates. This tooling allows CISA analysts to comb through security data in an automated fashion with mathematically and probabilistically based models to ensure high-fidelity anomalies are detected in a timely manner. ","This use case is inactive. Please see the ""intended purpose and expected benefits"" for a full description of this use case. ",Retired,,10/1/2022,,,N/A -Consolidated into another use case.,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Advanced Analytic Enabled Forensic Investigation,Department of Homeland Security,DHS,CISA,,,None of the above.,"CISA deploys forensic specialists to analyze cyber events at Federal Civilian Executive Branch (FCEB) departments and agencies, as well as other State, Local, Tribal, Territorial, and Critical Infrastructure partners. Forensic analysts can utilize advanced analytic tooling, in the form of Artificial Intelligence (AI) implementations, to better understand anomalies and potential threats. This can be further supported by the development of AI-assisted agents. Analysts can interact with such agents in a conversational manner to produce the input needed for complex search queries for behaviors or patterns in traffic that the agent then rapidly creates. Such tooling allows forensic specialists the capabilities to comb through event data in an automated fashion with mathematically and probabilistically based models to ensure high-fidelity anomalies are detected in a timely manner. ","This use case is inactive. Please see the ""intended purpose and expected benefits"" for a full description of this use case. ",Retired,,10/1/2022,,,N/A - Use case is in a pre-initation phase.,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Advanced Network Anomaly Alerting,Department of Homeland Security,DHS,CISA,,,None of the above.,"Threat hunting and Security Operations Center (SOC) analysts are provided terabytes per day of data from the National Cybersecurity Protection System's (NCPS’s) Einstein sensors. Manually developed detection alerts and automatic correlation via off the shelf tooling are common, but not comprehensive. Many network attacks can be probabilistically determined given sufficient training data and time. Analysts use automated tooling to further refine the alerts they receive and produce additional automated alerts based on aggregated information and backed in subject matter expertise. This tooling allows CISA analysts the capabilities to comb through data in an automated fashion with mathematically and probabilistically based models to ensure high-fidelity anomalies are detected in a timely manner.","This use case is inactive. Please see the ""intended purpose and expected benefits"" for a full description of this use case. ",Retired,,10/1/2022,,,N/A -Consolidated into another use case.,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Cyber Threat Intelligence Feed Correlation,Department of Homeland Security,DHS,CISA,,,None of the above.,"Cyber Data Feed Correlation uses AI enabled capabilities to provide accelerated correlation across multiple incoming information feeds. This enables more timely enrichment to improve the externally shared information feeds. AI allows the algorithm to use the information items and results to learn the most efficient ways to perform the task. Additionally, tailored algorithms could be created to provided sustained surveillance of threat actor Tactics, Techniques, and Procedures (TTPs).","This use case is inactive. Please see the ""intended purpose and expected benefits"" for a full description of this use case. ",Retired,,10/1/2022,,,N/A - Use case is in a pre-initation phase.,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Cyber Incident Reporting,Department of Homeland Security,DHS,CISA,,,None of the above.,"Cyber incident handling specialists utilize advanced automation tools to process data received through various threat intelligence and cyber incident channels. These tools leverage machine learning (ML) and Natural Language Processing (NLP) to increase the accuracy and relevance of data filtered and presented to human analysts and decision-makers. ML techniques also assist in aggregating the information in reports for presentation and further analysis, including data received through covered Cyber Incident Reporting for Critical Infrastructure Act (CIRCIA) entities. Such tools can leverage current dashboards and reports to cluster vulnerabilities by entity and sector, identifying trends in exposure and remediation. These solutions can also correlate vulnerabilities to existing frameworks (such as MITRE Adversarial Tactics, Techniques, and Common Knowledge (ATT&CK)) and other incidents to understand attackers’ techniques and improve modeling.","This use case is inactive. Please see the ""intended purpose and expected benefits"" for a full description of this use case. ",Retired,,10/1/2022,,,N/A -Consolidated into another use case.,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Cyber Vulnerability Reporting,Department of Homeland Security,DHS,CISA,,,None of the above.,"Vulnerability analysts require advanced automation tools to process data received through various vulnerability reporting channels, as well as aggregate the information for automated sharing. These tools leverage machine learning (ML) and Natural Language Processing (NLP) to increase the accuracy and relevance of data that is filtered and presented to human analysts and decision-makers. ML techniques also assist to aggregate the information in reports for presentation and further analysis. This includes data in the Known Exploited Vulnerabilities (KEV) and Common Vulnerabilities and Exposures (CVE) databases.","This use case is inactive. Please see the ""intended purpose and expected benefits"" for a full description of this use case. ",Retired,,10/1/2022,,,N/A - Use case is in a pre-initation phase.,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, AI Security and Robustness,Department of Homeland Security,DHS,CISA,,,None of the above.,"Frameworks, processes, and testing tools are developed to govern the acquisition, development, deployment, and maintenance of AI technologies. Technology integrators within CISA, as well as the rest of the federal enterprise, use AI-enhanced tools to ensure the trustworthy, robust, and secure operation of their AI systems. These tools use machine learning (ML) and Natural Language Processing (NLP) to enhance the assessment of AI technology within the agency by speeding up data processing. ","This use case is inactive. Please see the ""intended purpose and expected benefits"" for a full description of this use case. ",Retired,,10/1/2022,,,N/A - Active as R&D only.,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Operational Activities Explorer,Department of Homeland Security,DHS,CISA,,,None of the above.,"The integration of an AI tool into a dashboard to analyze cyber and physical incidents affecting critical infrastructure from all-source reporting could assist with maintaining operational clarity and centralized situational awareness. However, CISA decided not to move forward from the conceptual phase due to the nuances of event data and resource constraints. ","This use case is inactive. Please see the ""intended purpose and expected benefits"" for a full description of this use case. ",Retired,,10/1/2021,,,N/A - This use case was never initiated and never formally retired.,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Security Operation Center (SOC) Network Anomaly Detection,Department of Homeland Security,DHS,CISA,Mission-Enabling,,None of the above.,"This use case delivers improved internal government tools for hunting and detection of malicious threat actors on federal civilian networks. It automates manual data fusion and correlation processes and highlights potential anomalies, allowing CISA analysts to focus more on hunting adversaries. ","An interface is provided for analysts to query cybersecurity data, and dashboards are provided with potential cybersecurity alerts, including anomalies detected through predictive models and rule-based heuristics. ",Operation and Maintenance,Neither,9/4/2020,9/4/2020,8/22/2023,,Developed in-house.,,No,,,No,,No,Yes,Yes,,"Cybersecurity cloud, network and host logs, Cybersecurity threat intelligence (CTI) ","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Cyber Analytics and Data System Analytics Environment,More than 12 months,No,,Yes,,No,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Automated Detection of Personally Identifiable Information (PII) in Cybersecurity Data,Department of Homeland Security,DHS,CISA,Mission-Enabling,,None of the above.,"Automated PII Detection and Review Process uses analytics to identify and manage potential PII in submissions. If PII is flagged, the submission is sent to CISA analysts, who are guided by AI to review and confirm or reject the detection, redacting information if necessary. Privacy experts monitor the system and provide feedback. The system learns from this feedback, ensuring compliance with privacy regulations and improving efficiency by reducing false positives. Regular audits ensure the process remains trustworthy and effective. ","The system sends flagged data rows with potential PII to humans for review. ",Operation and Maintenance,Neither,2/1/2020,9/1/2020,12/1/2020,,Developed with contracting resources.,70QS0124C00000002,No,,,No,,Yes,Yes,Yes,,"Cybersecurity indicators of compromise (IOCs), Cybersecurity threat intelligence (CTI) ","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Trusted Automated eXchange of Indicator Information server,Less than 6 months,No,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Confidence Scoring for Cybersecurity Threat Indicators,Department of Homeland Security,DHS,CISA,Mission-Enabling,,None of the above.,"The confidence scores produced by this use case via an AI-driven decision tree process allow CISA's Automated Indicator Sharing (AIS) partners to contextualize the indicator information they are receiving so that they can ingest the data accordingly on their systems. ","A set of confidence scores is included along with the other fields in the indicator data set. ",Operation and Maintenance,Neither,6/1/2020,7/1/2020,7/1/2021,,Developed with contracting resources.,70QS0124C00000002,No,,,Yes,Data is shared only with government and critical infrastructure partners that have signed CISA's sharing agreements for the AIS program. IQA concerns are handled through that process.,Yes,Yes,Yes,,"Cybersecurity indicators of compromise (IOCs), Cybersecurity threat intelligence (CTI) ","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Trusted Automated eXchange of Indicator Information Server,Less than 6 months,No,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Report Analysis and Archive System (RAAS),Department of Homeland Security,DHS,CWMD,Mission-Enabling,,None of the above.,"To assist users of the Test and Evaluation document archive in locating documents and portions of document that are relevant for their needs by specifying individual queries as natural language questions, responding with detailed, natural language answers, and facilitating attainment of related or more precise information through follow-up questions.",Intermediate outputs include context summaries to facilitate indexing of document passages. Output to the user includes summarization of content relevant to the questions and interests indicated by the conversation with the user.,Acquisition and/or Development,Neither,8/1/2023,10/1/2023,,,Developed with contracting resources.,70RWMD20K00000016,No,,,,,,,No,,The data set available to the model is the set of CWMD Test and Evaluation Division test reports stored in the RAAS archive.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,,,,Yes,Report Analysis and Archive System (RAAS),Less than 6 months,No,,No,,Other,N/A,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, RelativityOne,Department of Homeland Security,DHS,DHS,Mission-Enabling,,None of the above.,"RelativityOne is a document review platform used to gain efficiencies in document review in litigation, FOIA, and other arenas where large-scale document review and production is necessary. It is currently in use across several different DHS Components and Offices.",Review Center tailors the review process by predicting the categorization of documents based on the reviewer's inputs.,Operation and Maintenance,Neither,5/1/2021,5/1/2021,8/1/2022,,Developed with contracting resources.,Purchased as COTS in several Components.,No,,,No,,Yes,Yes,No,,"N/A - The Agency does not own any of the data used to train, fine-tune, and/or evaluate performance of the model(s) used in this use case.","Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,No,No – agency does not have access to source code.,,Yes,Multiple across DHS,6-12 months,No,,No,,Other,N/A,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Commercial Generative AI for Text Generation (AI Chatbot),Department of Homeland Security,DHS,DHS,Mission-Enabling,,Improving the quality of written communications using AI tools.,"DHS employees are permitted to use commercially available generative AI for text generation in day-to-day work. Commonly referred to as AI Chatbots, these tools are able to dynamically create written content through text prompts submitted by the user. They use Natural Language Processing (NLP) and Large Language Models (LLM) to produce natural sounding language in a wide variety of contexts and styles. Approved applications of commercial Gen AI tools to DHS business include generating first drafts of documents that a human would subsequently review, conducting and synthesizing research on open-source information, and developing briefing materials or preparing for meetings and events.","Outputs may include writing, answers to question, and search results, depending on inputs from the user. ",Operation and Maintenance,Neither,10/24/2023,10/24/2023,10/24/2023,,,,,,,,,,,,,,,,,,No – agency does not have access to source code.,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Commercial Generative AI for Image Generation,Department of Homeland Security,DHS,DHS,Mission-Enabling,,Creating visually appealing presentations using AI-driven design suggestions.,"DHS employees are permitted to use commercially available generative AI for image generation in day-to-day work. These tools are able to dynamically create graphical content through text prompts submitted by the user. They use Natural Language Processing (NLP), in conjunction with other Machine Learning techniques such as Generative Adversarial Networks (GANs) and Diffusion Models, to produce images in a wide variety of contexts and styles.","Outputs may include pictures in different stylesk and types, depending on the inputs from the user. ",Operation and Maintenance,Neither,10/24/2023,10/24/2023,10/24/2023,,,,,,,,,,,,,,,,,,No – agency does not have access to source code.,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Commercial Generative AI for Code Generation,Department of Homeland Security,DHS,DHS,Mission-Enabling,,Improving the quality of written communications using AI tools.,DHS employees are permitted to use commercially available generative AI for code generation in day-to-day work. These tools are able to dynamically create usable code through plain-language prompts submitted by the user. They use Natural Language Processing (NLP) and Large Language Models (LLM) to produce code in many different programming languages and for a variety of tasks.,"Outputs include useable code in different programming languages, depending on the inputs from the user.",Operation and Maintenance,Neither,10/24/2023,10/24/2023,10/24/2023,,,,,,,,,,,,,,,,,,No – agency does not have access to source code.,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, OCFO Code Assist GPT,Department of Homeland Security,DHS,FEMA,Mission-Enabling,,None of the above.,"1) Code Assist GPT is an internal facing GenaAI tool to augment the FEMA workforce in generating and troubleshooting existing queries in established query languages. 2) The tool provides improved queries and rapid iteration to produce desired scripts/query language saving the end user 80-90% of the time versus custom query development. The tool also opens other computer languages, as required, to support our data analytics community."" ","Code Assist GPT is an internal facing GenaAI tool to augment the FEMA workforce in generating and troubleshooting existing queries in established query languages. The user simply enters the language they are trying to query (i.e. SQL, Java, COBOL, etc) and the Code Assist GPT then provides a proposed query based on the elements provided by the user. If the user experiences an error or the query is not successful, the Code Assist GPT maintains the session as long as the user is still logged in and hasn't restarted the session, the user can continue to prompt the GPT to provide enhancements to the provided query until results are as expected. At the end of the user session, all prompts/queries are removed and no data is stored outside of the active user session. ",Implementation and Assessment,Neither,6/1/2024,10/1/2024,,,Developed with contracting resources.,70FA3124F00000027,No,,,No,,No,Yes,No,,"N/A - The Agency does not own any of the data used to train, fine-tune, and/or evaluate performance of the model(s) used in this use case.","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,FEMA OCFO GPT,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Geospatial Damage Assessments,Department of Homeland Security,DHS,FEMA,Emergency Management,,None of the above.,"We will integrate these automated tools for imagery-based damage assessments into RGO processes and systems to inform FEMA geospatial analysts. FEMA geospatial analysts will review output and develop recommendations, so FEMA can quickly detect and characterize damaged and undamaged buildings, identify the concentrations of damage, identify and detect debris, and to support rapid response and recovery activities.","AI models generate points or polygons over areas where damage or debris is likely. This output is a recommendation meant to trigger investigation by human analysts via traditional damage assessments. In large scale incidents, imagery collection can be comprised of hundreds of thousands of miles of land and millions of structures. AI automation allows for the prioritization of imagery exploitation where damage is likely to have occurred, reducing the time required to identify impacts and summarize damage sustained. ",Implementation and Assessment,Neither,8/1/2022,8/1/2022,,,Developed with contracting resources.,The PIID is specific to the disaster the use case is used for.,No,,,No,,No,Yes,Yes,,Geospatial Damage Assessments.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,"We are not developing models in house, we have contracted out our AI work",Other,"We are not developing models in house, we have contracted out our AI work",Other,"We are not developing models in house, we have contracted out our AI work",None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Hazard Mitigation Assistance Chatbot,Department of Homeland Security,DHS,FEMA,Emergency Management,,None of the above.,"The intended purpose is to deploy an AI-powered internal chatbot that serves as a centralized, intuitive user interface for staff across all FEMA regions and headquarters. The intended outcomes include: Improving Efficiency and Consistency by providing quick access to accurate information, the chatbot streamlines work processes, reduces redundancy, and ensures consistent procedures across regions. Accelerate Onboarding of new hires become competent more quickly, reducing the time to full productivity and strengthening the workforce. Ensure Compliance and Accountability by being Transparent and accurate guidance promotes adherence to HMA policies, reducing errors and enhancing program integrity. Enhancing Decision-Making by providing access to precise citations and real-time data improves the quality of decisions, benefiting program outcomes. Supporting FEMA's mission by empowering staff to perform effectively, the chatbot enhances overall efficiency and service delivery, aligning with FEMA's goal of improving program delivery and serving the whole community.","The chatbot will provide accurate, uniform answers with citations, ensuring everyone is aligned and reducing the risk of misinterpretation of publicly available HMA data sourced from FEMA.gov. The chatbot will provide clear, plain-language explanation, highlighting eligibility criteria, funding priorities, and application processes for each HMA program. The chatbot will quickly retrieve relevant HMA policy documents, provides precise citations, and summarizes key points. The chatbot will compile publicly available of publicly available HMA data sourced from FEMA.gov., perform the analyses, and present it in a clear report with visualizations.",Acquisition and/or Development,Neither,9/5/2024,9/5/2024,,,Developed with both contracting and in-house resources.,70FA6022A00000001,No,,,,,,,Yes,,"The agency-owned data included all HMA policy, training, and data available on FEMA.gov. This include OpenFEMA datasets, Legal/Policy Documents (i.e., federal legal and policy references, such as nondiscrimination clauses and presidential memorandums); Regulations (i.e., regulatory texts, including sections of the Code of Federal Regulations (CFR)); and Guides and Handbooks (i.e., FEMA-issued guides that provide frameworks and instructions, such as planning handbooks and operational guides).","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,,,,No,,Less than 6 months,Other,We have identified the FEMA GenAI Sandbox as the IT infrastructure and are working with OCIO to utilize the infrastructure.,Other,We are working with OCIO to develop a process to request the Azure computing resources to develop and test the chatbot.,Other,"Yes, but Mission requirements and resource restraints are a factor.",None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Planning Assistant for Resilient Communities (PARC),Department of Homeland Security,DHS,FEMA,Emergency Management,,None of the above.,"Hazard mitigation plans are not only a foundational step that communities can take to build their resilience but can be lengthy to produce and challenging for communities that lack resources to do so. PARC will specifically support State, Local, Tribal, and Territorial (SLTT) governments’ understanding of how to craft a plan that identifies risks and mitigation strategies as well as generate draft plan elements—from publicly-available, well-researched sources — that governments could customize to meet their needs. This capability could provide planners more time to allocate toward community engagement and stakeholder collaboration, and lead to more communities having the ability to submit grant applications for funding to become more resilient and reduce disaster risks.","The Beta Release GenAI Plan Generator (OpenAI GPT-4o) is an LLM that generates sections of the Hazard Mitigation Plans based on user inputs. It processes prompts and user responses to create detailed, regulatory-compliant Hazard Mitigation Plan sections. Beta Release PARC Assistant (OpenAI GPT-4o) prodices responses through an interactive chat assistant. The Assistant helps planners by answering questions related to the Hazard Mitigation Plan process, offering explanations of policy guidelines, and guiding users through FEMA’s regulatory requirements.",Acquisition and/or Development,Neither,10/27/2023,1/29/2024,,,Developed with both contracting and in-house resources.,"70FA4020A0000000281, 70FA3020A00000008",No,,,,,,,Yes,,"The agency-owned data included: Legal/Policy Documents (i.e., federal legal and policy references, such as nondiscrimination clauses and presidential memorandums); Regulations (i.e., regulatory texts, including sections of the Code of Federal Regulations (CFR)); and Guides and Handbooks (i.e., FEMA-issued guides that provide frameworks and instructions, such as planning handbooks and operational guides).","Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,,,,No,,6-12 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Recovery and Resilience Resource (RRR) Portal,Department of Homeland Security,DHS,FEMA,Government Services (includes Benefits and Service Delivery),,None of the above.,"The AI-layer (currently unfunded) would include a smart matching wizard functionallity via AI large language model (LLM) to connect SLTT partners with the resources that meet their unique needs. Increasing access and improving navigation of federal resources will benefit a wide array of stakeholders across all sectors, including private and public. The primary stakeholders are SLTTs that are looking to support recovery and/or resilience in communities. This includes 50 States, DC, and 5 US Territories, about 80,000 local governments (including special districts), 574 Federally recognized Tribal governments, and their partners. Additional stakeholders include the 25+ federal departments and agencies that are members of the Mitigation Framework Leadership Group (MitFLG), who have equity in the availability of comprehensive resources, best practices, or tools for SLTTs to recover from or increase resilience to disasters. ","The smart matching wizard will be an AI LLM that analyzes user search behavior and matches recommended resources from across the expansive data library that will include resources from federal, state, local, nonprofit sectors.",Initiated,Neither,4/30/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Digital Processing Procedure Manual (D-PPM),Department of Homeland Security,DHS,FEMA,Mission-Enabling,,None of the above.,"The AI addresses the inefficiency in the current manual search process, where agents spend significant time navigating static documents like PDFs (2000+ pages/70 documents) to answer survivor questions. As policies can change frequently, especially during disasters, it is crucial that agents access the most up-to-date information quickly and accurately. Expected Benefits and Positive Outcomes: Faster Response Times: The AI will enable agents to retrieve information quickly, reducing the overall time spent on each call. Improved Answer Accuracy: By providing more accurate and relevant search results, the AI will help agents deliver consistent, correct responses. Cost Savings: Reducing the time spent per call will improve operational efficiency, leading to potential cost savings. Reduced Escalations: By empowering agents with the correct information, the tool will lower the frequency of escalations to supervisors, allowing them to focus on more complex cases. Increased Capacity for Complex Casework: The time saved through the AI tool will allow agents to handle more complex queries, ultimately improving service quality and survivor satisfaction. Scalability: As the AI tool is rolled out, it will provide a scalable solution that can grow with the agency’s needs during large-scale disaster responses.","The AI will provide search results based on the agent's query, identifying the most relevant policies, procedures, and disaster-specific information. These results will be returned in real-time, enabling agents to quickly review and apply the information to their responses. While the AI does not make final decisions, it assists agents by highlighting the most applicable information for them to review. Search Results: The AI use case will pull information on relevant policies and procedures from the prescribed policy inventory. It may generate a summary of the search results with citations of the source documents.",Initiated,Neither,10/2/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, FEMA OCFO GPT,Department of Homeland Security,DHS,FEMA,Mission-Enabling,,None of the above.,"FEMA OCFO GPT - B is an internal facing GenAI tool to augment the FEMA workforce in generating initial responses to questions for the record and providing a preliminary response to the Program Office to use in their formal response to the request. It provides draft responses reducing the data gathering stage and providing additional time for analysis, response, and approval. This has reduced the analyst initial level of effort versus individual research by 80% on initial surveys.""","FEMA OCFO GPT is an internal facing GenAI tool to augment the FEMA workforce in generating initial responses to questions for the record and providing a preliminary response to the Program Office to use in their formal response to the request. FEMA OCFO GPT leverages public facing and internal deliberative documents (hearing testimony, answers sent to previous questions, and other relevant documents) to assist in answering questions the Agency receives. The tool generates a draft response that is then refined/updated prior to providing a formal response. This tool is leveraged in the data gathering stage and does not replace any current analysts work or Agency leadership review prior to submittal via the formal request for information or question from an internal stakeholder or external stakeholder.",Retired,,7/1/2023,8/1/2023,2/12/2024,N/A -Consolidated into another use case.,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, OCFO Response Augmentation Suite,Department of Homeland Security,DHS,FEMA,Mission-Enabling,,None of the above.,"1) FEMA OCFO GPT - B is an internal facing GenAI tool to augment the FEMA workforce in generating initial responses to questions for the record and providing a preliminary response to the Program Office to use in their formal response to the request. 2) FEMA OCFO GPT provides draft responses reducing the data gathering stage and providing additional time for analysis, response, and approval. This has reduced the analyst initial level of effort versus individual research by 80% on initial surveys. 1) Travel Policy GPT is an internal facing GenAI tool to augment the FEMA workforce in generating initial responses to questions regarding FEMA/DHS Travel Policy, including the JTR, and providing a preliminary response to the travel specialist to use in their formal response to the queries. 2) The tool provides improved responses saving the end user 80-90% of the time versus regular engagement with the Travel Service Center. The tool also allows travelers to ask specific questions that require Travel Service Center engagement, limiting the needed triage and speeding resoultion times. 1) Fiscal Policy GPT is an internal facing GenAI tool to augment the FEMA workforce in providing preliminary responses to questions regarding FEMA/DHS Fiscal Policy. 2) The tool provides improved responses saving the end user 80-90% of the time versus regular engagement with the DHS/FEMA OCFO Policy. The tool also allows internal users to ask specific questions that require Fiscal Policy engagement, limiting the needed triage and speeding resoultion times.","FEMA OCFO GPT is an internal facing GenAI tool to augment the FEMA workforce in generating initial responses to questions for the record and providing a preliminary response to the Program Office to use in their formal response to the request. FEMA OCFO GPT leverages public facing and internal deliberative documents (hearing testimony, answers sent to previous questions, and other relevant documents) to assist in answering questions the Agency receives. The tool generates a draft response that is then refined/updated prior to providing a formal response. This tool is leveraged in the data gathering stage and does not replace any current analysts work or Agency leadership review prior to submittal via the formal request for information or question from an internal stakeholder or external stakeholder. Travel Policy GPT is an internal facing GenAI tool to augment the FEMA workforce in generating initial responses to questions regarding FEMA/DHS Travel Policy, including the JTR, and providing a preliminary response to the travel specialist to use in their formal response to the queries. Travel Policy GPT leverages the JTR and internal travel policy documents to assist in answering questions the FEMA Travel Service Center receives. The tool generates a draft response that is then refined/updated prior to providing a formal response. This tool is leveraged in the data gathering stage and does not replace any current analysts work or leadership review, as required, prior to submittal via the formal request for information or question from a traveler traveling on FEMA's behalf or in accordance with their duties to support the FEMA mission. Fiscal Policy GPT is an internal facing GenAI tool to augment the FEMA workforce in providing preliminary responses to questions regarding FEMA/DHS Fiscal Policy. Fiscal Policy GPT will leverage the DHS FMPM and FEMA Fiscal Policy documents. The tool is planned to generate a draft response with references to assist FEMA internal workforce in compliance with established policy. This tool is leveraged in the data gathering stage and does not replace good judgement or in any way impact decisions on compliance with established Fiscal Policy. The tool will display a disclaimer to this effect that will be cleared with FEMA/DHS OGC, as appropriate.",Operation and Maintenance,Neither,7/1/2023,8/1/2023,2/1/2024,,Developed with contracting resources.,70FA3124F00000027,No,,,No,,No,Yes,Yes,,"Budget Exhibits, Passback Materials, Hearing Testimony, Questions Received, Answers Provided Travel Policy Documents (Joint Travel Regulation (JTR), DHS Travel Policy, FEMA Travel Policy) Fiscal Policy Documents - Treasury Financial Manual (TFM), DHS Financial Policy Manual, FEMA Fiscal Policies","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,FEMA OCFO GPT,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Incident Management Workforce Deployment Model (depmod),Department of Homeland Security,DHS,FEMA,Mission-Enabling,,None of the above.,"The model is able to extract patterns from big data, combine these patterns in a way that mirrors a real-world process, and simulate likely outcomes to yield predictions in the form of relatable, mission-centric, metrics.",The system primarily outputs predictions that can then (optionally) be summarized and communicated as recommendations.,Operation and Maintenance,Neither,4/1/2021,8/1/2021,2/1/2022,,Developed with both contracting and in-house resources.,HHSN316201200112W,No,,,No,,No,No,Yes,,The Deployment Tracking System (DTS) is currently the primary data source. It provides a historical record of IM workforce deployments that can be validated for accuracy and used for model training and prediction.,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, Individual Assistance (IA) & Public Assistance (PA) Projections,Department of Homeland Security,DHS,FEMA,Government Services (includes Benefits and Service Delivery),,None of the above.,"The purpose of these supervised learning models is two-fold: 1. For informational purposes: the models will produce predictions for to-be-determined quantities of interest. These quantities are often of interest to Agency personnel in the field, region, and headquarters, as well as DHS, OMB, NSC, and the Whitehouse. 2. For decisional purposes: in addition to being informative, the model’s predictions are likely to be used for decision making. This may include implicit use (e.g., seeing that there are a large number of households that require assistance may motivate FEMA CFO to consider requesting additional funding for the Disaster Relief Fund) or explicit use (e.g., the number of estimated housing units required may be used to determine the number of housing units to order just-in-time). Anticipated benefits of using supervised learning models include: 1. Increased predictive performance: supervised learning models offer means of quantifying and choosing models with the best possible predictive performance. 2. Timely prediction: supervised learning models can produce reasonably high performance predictions based on limited data. 3. Quantification of uncertainty: supervised learning models are able to provide prediction intervals and predictive distributions in addition to point predictions to give users a sense of how uncertainty is inherent in the prediction. 4. Rigor: supervised learning models are well founded on statistical learning theory and provide a level of mathematical rigor that speaks to the reliability and limitations of methods.","These supervised learning models produce predictions, not recommendations and not decisions (though they can be used to inform human users in making recommendations and decisions). At minimum, these supervised learning models will produce point predictions for the different quantities of interest for disaster declarations. Additionally supervised learning models may produce prediction intervals or predictive distributions as feasible and appropriate for the given prediction problem. Often these outputs will be shared via business intelligence tools (e.g., Tableau or PowerBI) for wide internal FEMA use. Some predictions may be shared to a more restricted audience through simpler means (e.g., an excel workbook) as appropriate.",Operation and Maintenance,Neither,8/1/2018,8/1/2018,6/1/2024,,Developed in-house.,,Yes,Federal Emergency Management Agency – DHS ,Applying for and receiving disaster assistance,No,,No,No,Yes,,Data from the National Emergency Management Information System (NEMIS) and Grants Manager,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,No,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Voice Analytics for Investigative Data,Department of Homeland Security,DHS,ICE,Law & Justice,,None of the above.,"While conducting investigations, Homeland Security Investigations (HSI) encounters a significant volume of live and recorded content, requiring human language to be extracted and analyzed. Currently, this is achieved through manual processes that are extremely labor-intensive and expensive. To address this challenge, HSI has developed Voice Analytics, a cutting-edge technology that leverages machine learning and advanced speech processing techniques to perform multilingual speech transcription and translation on user-supplied audio files. This innovative solution enables HSI personnel to quickly identify new investigative leads and analyze the outputs against other law enforcement datasets, significantly improving the efficiency and effectiveness of their investigations compared to manual voice analytics processes.",This system outputs transcripted and translated audio files.,Implementation and Assessment,Neither,3/1/2020,7/1/2022,,,Developed with contracting resources.,"70OCTD22FR00000002, 70OCTD22FR00000235, 70OCTD22FR00000009",No,,,No,,Yes,No,Yes,,"N/A - The Agency does not own any of the data used to train, fine-tune, and/or evaluate performance of the model(s) used in this use case.","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",Sex/Gender,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Repository for Analytics in a Virtualized Environment,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Semantic Search and Summarization for Investigative Reports,Department of Homeland Security,DHS,ICE,Law & Justice,,None of the above.,"Effective investigations require querying large volumes of unstructured reports to extract and summarize contextually-relevant information. Current lexical search methods are insufficient for finding such information. Semantic Search and Summarization enables contextually-relevant information extraction and summarization from large volumes of unstructured investigative reports. The output will enhance and optimize investigative analysis by accelerating person-of-interest identification, surfacing trends, and improving network or fraud detection.","The output consists of contextually relevant contents of reports that can be used in investigations to accelerate person-of-interest identification, surface trends, and improve network or fraud detection.",Implementation and Assessment,Neither,10/1/2023,10/1/2023,,,Developed with contracting resources.,70OCTD22FR00000002; 70OCTD22FR00000235; 70OCTD22FR00000009,No,,,No,,Yes,Yes,Yes,,"HSI Report of Investigation (ROI) data. HSI ROIs document the findings and results of an investigation into a specific case or matter. ROIs typically include information about the investigation, such as individuals or organizations suspected of violating federal laws related to customs, immigration, or national security. ROIs may also include recommendations for further action, such as prosecution, deportation, or other enforcement measures.","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Virtual Agent Chatbot for Human Capital,Department of Homeland Security,DHS,ICE,Education & Workforce,,None of the above.,"The intended purpose of the ServiceNow Virtual Agent natural language understanding (NLU) is to facilitate seamless interactions between users and the chatbot. It aims to provide automated support for common Human Capital related questions and inquiries, enhancing efficiency and user experience. By understanding natural language, the virtual agent can provide relevant and accurate responses, making it easier for users to get the help they need. Reduce the amount of submitted inquiries that need to be manually processed by OHC. Reduce the amounts of clicks it takes for the user to find the information they are looking for.",The natural language understanding (NLU) will point towards pre-defined Human Capital related categories that are most relevant based on the users’ inquiries. Specific responses/answers to user questions and resources associated with the questions if applicable.,Implementation and Assessment,Neither,5/9/2024,8/5/2024,,,Developed with contracting resources.,70CTD023FR0000134,No,,,No,,No,No,Yes,,ICE Office of Human Capital team members compiled documentation to train the NLU model based on their subject matter expertise on human capital related questions and inquiries. The data was compiled from knowledge base articles and frequently asked questions for human capital related inquiries.,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,ServiceNow,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Machine Learning Translation Technology Initiative,Department of Homeland Security,DHS,ICE,Law & Justice,,None of the above.,"The solution will be used to facilitate non-critical communications only. When deployed to production, the solution should not be used alone, absent human quality control, when materials are vital to an individual’s rights or benefits, or when the source materials contain non-literal language (e.g., slang, metaphor), lack clear grammar or structure, contain abbreviations or acronyms, or are overly complex, technical, or wordy. If any interactions indicate the presence of legal or medical issues or a need for greater accuracy, ERO personnel are required to seek additional language assistance via bilingual staff or a professional language line. The solution was first spearheaded by U.S. Coast Guard for maritime usage with the support of DHS Science and Technology Directorate through the Silicon Valley Innovation Program (SVIP). ICE ERO received FY22 funding to initiate the proof of concept with the vendors vetted by USCG and DHS S&T through an interagency agreement. Solutions, in the form of mobile phone applications, are being developed under research and development phases and projected to completely meet ERO’s requirements by January 2026.","The Machine Learning Translation Technology Initiative will offer real-time communication/translation services to assist personnel in providing meaningful access to programs and services for limited English proficient (LEP) noncitizens. AI will be used to provide voice to text, text to voice, and voice to voice translations in at least 21 languages.",Acquisition and/or Development,Neither,7/1/2022,7/1/2022,,,Developed with contracting resources.,Mylanguage Phase 3 - 70RSAT23T00000030; Kynamics Phase 4 - 70RSAT24T00000029,No,,,,,,,No,,"N/A - The Agency does not own any of the data used to train, fine-tune, and/or evaluate performance of the model(s) used in this use case.","Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,,,,No,,More than 12 months,No,,No,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Policy Analyst Assistant,Department of Homeland Security,DHS,ICE,Government Services (includes Benefits and Service Delivery),,None of the above.,"The Student and Exchange Visitor Program (SEVP) uses AI to help policy analysts quickly find and summarize information about visa rules for students and schools. This saves time and helps analysts respond faster to questions. This project is intended purpose is to enable SEVP Policy staff to provides responses to policy related inquiries and conduct policy research and analysis: It will be a tool embedded in workflow processes. The processes augmented by the AI includes responses for questions asked by SEVP internal staff and responses given to schools and students by SEVP personnel. For this process, the source of information for the responses will be from authoritative, pre-vetted documents. This reduces the workload on the Policy staff and allows them to focus on more complex policy and guidance issue. For these processes, the AI will assist with researching internal documents and information from regulations, applicable legislation, and other federal guidance. It will assist the policy analyst with summaries, citations, and initial drafts. The benefits includes faster, more consistent responses from Policy and the ability to handle additional work without adding staff.",The outputs provides answers to frequently asked policy and guidance questions and summaries and drafts for more in-depth analysis of policy issues. All responses go to an SEVP staff member and are not the final work product.,Acquisition and/or Development,Neither,8/1/2024,10/14/2024,,,Developed with contracting resources.,70CTD020FR0000006,No,,,,,,,Yes,,"We are not training the AI, but we are augmenting the AI using RAG (Retrieval Augmented Generation) dataset with SEVP policy documents. These will be used to generate responses and the data will also be to evaluate the answers given.","Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,,,,Yes,SEVPAMS,Less than 6 months,Yes,,No,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, SEVP Response Center Chatbot - SID (SEVIS Interactive Dialog),Department of Homeland Security,DHS,ICE,Government Services (includes Benefits and Service Delivery),,None of the above.,"AI provides SID the ability to understand voices and its deterministic question and answer workflow (1) enables SID to answer routine caller questions without a help desk agent, and (2) when a help desk agent is required, SID will create a ticket with a caller transcript to reduce the burden on the agent. This frees up the human agents to deal with more complex cases and issues with specific records.","SID answers frequently asked questions from callers. If the SID cannot answer a caller’s question, it turns the caller over to an agent in the response center. The chatbot captures the interaction with the caller and sends the information via an API to Student and Exchange Visitor Program Automated Management System (SEVPAMS).",Acquisition and/or Development,Neither,2/6/2023,3/6/2023,,,Developed with contracting resources.,PIID 70CTD020FR0000006/70CTD022FR0000005,No,,,,,,,Yes,,"N/A - The Agency does not own any of the data used to train, fine-tune, and/or evaluate performance of the model(s) used in this use case.","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,,,,Yes,SEVPAMS,Less than 6 months,Yes,,No,,No,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Title III Semantic Search and Summarization for Translated Content,Department of Homeland Security,DHS,ICE,Law & Justice,,None of the above.,"The Title III Semantic Search and Summarization functionality will augment translation and transcription services by extracting relevant data using machine learning and natural language processing for correlation and semantic search. Results can then be summarized using a large language model, giving users a tool to target relevant data only. This capability accelerates investigative analysis by rapidly identifying persons of interest, surfacing trends, and detecting networks or fraud, saving hundreds of man hours in manual analysis. HSI will use this tool to generate leads, and further action will be required by investigators and analysts as part of the full investigative process before any action is taken against an individual.","This will allow for the summarization of large data sets using a large language model, giving users a tool to target relevant data only.",Initiated,Neither,4/12/2022,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Digital Records Manager (DRM) User Assistance Chatbot,Department of Homeland Security,DHS,ICE,Mission-Enabling,,None of the above.,"This is an AI records search tool to help investigators more efficiently search and gather information. The Digital Records Manager (DRM) User Assistance Chatbot is intended to increase user efficiency by providing answers to commonly asked question without the need to manually refer to documentation or submit a help desk ticket. A reduction in the volume of submitted help desk tickets is expected as a result.","The outputs of the DRM User Assistance Chatbot will be natural language responses to user questions, based on a custom Knowledge Base of DRM documentation artifacts, supported by the natural language capabilities of the backing LLM.",Initiated,Neither,9/10/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Burlington Finance Center Voice Bot,Department of Homeland Security,DHS,ICE,Government Services (includes Benefits and Service Delivery),,None of the above.,"Identify and verify the caller, retrieve the status of the bond, and share the bond status with the requester and/or answer administrative FAQs.","It will use Natural Language Understanding (NLU)/Natural Language Processing (NLP) to perform voice-to-text and text-to-voice translations, giving it the ability to recognize voices and meaning. The BFC Voice Bot will be deterministic and will not use Generative AI.",Initiated,Neither,9/3/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, RAVEn Compliance Automation Tool (CAT),Department of Homeland Security,DHS,ICE,,,None of the above.,Repository for Analytics in a Virtualized Environment (RAVEn) Compliance Automation Tool (CAT) is being developed as part of an effort to modernize HSI’s Form I-9 Inspection Process. The goal is to uses machine learning (ML) and automation to increase the speed and efficiency of ingesting and processing Forms I-9 data. Easy to use front-end interface workflow that increases work productivity and reduces manual entry. RAVEn CAT currently employs an Optical Recognition Service (OCR) model and software (Tesseract OCR) to identify pixel coordinates of handwritten and read/extract computer typed characters from ingested forms for processing. Additional research into opensource ML Object Detection models is being made to help further augment accuracy of text identification and extraction of ingested forms into the pipeline. ,"This use case is inactive. Please see the ""intended purpose and expected benefits"" for a full description of this use case. ",Retired,,,,,7/1/2022,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Data Tagging and Classification,Department of Homeland Security,DHS,ICE,,,None of the above.,"The Homeland Security Investigations (HSI) Innovation Lab is developing an analytical platform called the Repository for Analytics in a Virtualized Environment (RAVEn). RAVEn facilitates large, complex analytical projects to support ICE’s mission to enforce and investigate violations of U.S. criminal, civil, and administrative laws. RAVEn also enables users to develop new tools to analyze trends and isolate criminal patterns as HSI mission needs arise. For more information, please read the DHS/ICE/PIA-055 - Privacy Impact Assessment 055 for the Repository for Analytics in a Virtualized Environment (RAVEn).  RAVEn leverages data tagging and classification to do the following: The Email Analytics Tool streamlines how special agents and criminal analysts search, filter, translate, and report on electronic communications evidence and will help investigators more effectively determine the structure and organization of criminal enterprises. The RAVEn - Lead Tracker is a centralized system where agents can send and receive leads and enter outcomes such as arrests and seizures. The goal is for all leads in the agency to be found in one place, rather than in various email inboxes. The overarching goal of Mobile Device Analytics is to improve the efficiency of agents and analysts in identifying pertinent evidence, relationships, and criminal networks from data extracted from mobile devices.","This use case is inactive. Please see the ""intended purpose and expected benefits"" for a full description of this use case. ",Retired,,,,,N/A - Deconsolidation of several use cases,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Facial Recognition Service,Department of Homeland Security,DHS,ICE,,,None of the above.,"The Facial Recognition Service is used during investigations conducted by Homeland Security Investigations (HSI) agents and analysts for identification of known individuals, as well as extracting faces for further investigations from perpetrators including child exploitation offenses, human rights atrocities, and war criminals.  This is a DHS HSI Innovation Lab/RAVEn project. The Repository for Analytics in a Virtualized Environment (RAVEn) facilitates large, complex analytical projects to support ICE’s mission to enforce and investigate violations of U.S. criminal, civil, and administrative laws. RAVEn also enables tools used to analyze trends and isolate criminal patterns as HSI mission needs arise.","This use case is inactive. Please see the ""intended purpose and expected benefits"" for a full description of this use case. ",Retired,,8/1/2020,12/1/2020,8/1/2021,N/A - Deconsolidation of several use cases,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, "Machine Translation (Previously Language Translator)",Department of Homeland Security,DHS,ICE,,,None of the above.,"The Rapid Analysis and Visualization Engine for Reporting (RAVEn) platform provides capabilities to upload data that may contain text in languages other than English. To support the agent/analyst analysis of the ingested data as well as to provide for the means to standardize searches across RAVEn data sources, RAVEn provides language translation capabilities to create text conversions between the original language and English. Data translation can be performed on ingest or on an ad hoc basis. Solution will provide translation support of, at minimum, plain text, word documents, and portable document formats using Natural Language Processing and machine learning.","This use case is inactive. Please see the ""intended purpose and expected benefits"" for a full description of this use case. ",Retired,,2/1/2021,8/1/2019,8/1/2022,N/A -Consolidated into another use case.,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Investigative Prioritization Aggregator,Department of Homeland Security,DHS,ICE,Law & Justice,,None of the above.,"The sheer volume of data associated with investigations often overwhelms human capabilities, making it challenging for HSI personnel to analyze evidence and identify key players in criminal networks. Currently, there is no effective mechanism to quantify the level of evidence related to a particular subject or entity, or to determine which actors within a network are the most influential. This is particularly critical in the context of the counter-opioid/fentanyl mission, where timely and accurate intelligence is essential. To address this challenge, this project utilizes machine learning to assign point values to data, enabling the scoring of information associated with a given selector, such as a phone number or legal name. This scoring system helps to understand the importance of an entity to investigations and the potential consequences of removing or neutralizing that entity. By doing so, HSI personnel can focus on high-priority targets and associated criminal networks, ultimately enhancing their ability to disrupt and dismantle these threats.",The output is scored entity data (such as a phone number or legal name). This scoring system helps to understand the importance of an entity to investigations and the potential consequences of removing or neutralizing that entity.,Operation and Maintenance,Neither,2/1/2021,10/1/2022,2/1/2024,,Developed with contracting resources.,196918HSI00013,No,,,No,,Yes,Yes,Yes,,Law Enforcement Sensitive (LES) investigative data.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Repository for Analytics in a Virtualized Environment,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Hurricane Score,Department of Homeland Security,DHS,ICE,Law & Justice,,None of the above.,"The Immigration and Customs Enforcement (ICE) Enforcement and Removal Operations (ERO) division would like to explore technology that could help lead to time savings and eliminate human error due to overburdened officers. An AI generated hurricane score could also minimize the risk that a factor is overlooked due to officers being overburdened. The hurricane score may be one factor that is used to inform the decision-making process. The “Hurricane Score” models the potential risk (1-5) that a noncitizen who is released from detention with the requirement to check in with ICE through monitoring will abscond from the program, with a higher number indicating a higher risk of absconding.","The Hurricane score model determines a probability of absconding and outputs a number (1-5), with a higher number indicating a higher risk of absconding. This number is used as one of many factors an officer uses when reviewing an ATD-ISAP case to determine the appropriate level of case management and technology for a noncitizen.",Operation and Maintenance,"Rights-Impacting ",2/1/2017,2/1/2017,2/1/2019,,Developed in-house.,,No,,,No,,Yes,No,No,,Inactive case data from individuals enrolled in the ATD-ISAP program.,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ","Sex/Gender,Age,Other",Country of Origin,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,Yes,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.","There is a risk that the Absconder Model may include false positives and model bias. A false positive would occur when non-absconders are incorrectly classified with a score indicating a high probability of absconding. Model bias can arise if the training data is not representative of the full range of participants, leading to poor performance on certain patterns or characteristics. These risks are identified and if necessary remediated through error analysis during model testing and model performance monitoring. ",Yes – by the CAIO,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,N/A - individuals are not interacting with the AI for this use case,,No,It is reevaluated and retrained on an annual basis.,Direct user testing,,Yes,,Other,H-Score adds one piece of info to a larger profile for decision-making by a human. ICE Mobile Check-in Application,Department of Homeland Security,DHS,ICE,Law & Justice,,None of the above.,"The Check-In App is an opt-in capability that allows for low-risk adult noncitizens to securely check-in via an ICE Verified user account, verify their identity, identify their location, and satisfy check-in requirements thereby reducing in-person office visits and reducing manual processing work. AI-enabled capabilities within the ICE Check-In application give ICE the ability to scale its interactions to a greater number of noncitizens and reduce in-office presence at ICE facilities.","AI/ML is used for this face detection and tracking. Once captured, the photo is verified with a 1:1 verification using the CBP Traveler Verification Service. Verification of all requirements will satisfy check-in requirements. A failed verification will result in a review by an officer. The officer can request the user try again or request the user use the default in-person check-in process by making an appointment with their local ICE office.",Operation and Maintenance,"Rights-Impacting ",7/10/2023,10/2/2023,11/30/2023,,Developed with contracting resources.,70CTD022D00000010,No,,,No,,Yes,Yes,Yes,,"N/A - The Agency does not own any of the data used to train, fine-tune, and/or evaluate performance of the model(s) used in this use case.","Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,ICE Portal,6-12 months,Yes,,Yes,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,Yes,"Benchmark evaluation: Testing of the AI model has involved the use of benchmarks (either publicly available or internally created) to estimate performance in real-world settings, but has not been tested in an operational environment. ","There is a risk that the face detection model testing will show the model performs poorly for certain performance metrics that cannot be remediated through software configuration or other reasonable means. In that scenario, ICE will either pursue using a different face detection model or pursue additional testing to uncover the root cause of poor performance.",Yes – by the CAIO,No monitoring protocols have been established: Necessary infrastructure needed to perform monitoring of AI systems in production is not available and/or a plan to monitor models has not yet been established.,No - Some individual decisions or actions require direct human oversight.,,None of the above,,No,"None for liveness detection using Google ML Kit. For facial verification, ICE leverages CBP TVS and inherits their controls.",Direct user testing,,Yes,,Yes, AI Assisted Compromise Email Detector (AACED),Department of Homeland Security,DHS,ICE,Mission-Enabling,,None of the above.,"The use case was developed to assist ICE SOC in reviewing a collection of emails between ICE personnel and Microsoft that were part of ED 24-02. The use case provides a faster mechanism to the SOC analysts to determine indicators of compromise, reducing the level of effort for these individuals’ analysis exponentially. To assist the analysts, Named Entity Recognition (NER) was used to detect PII and other associated keywords to increase analyst productivity, and reduce time required to analyze emails. ",Outputs are named entities and generated text for specific questions. Chat interface for analyst to conduct Q&A with email as context.,Operation and Maintenance,Neither,5/1/2024,5/1/2024,6/1/2024,,Developed in-house.,,No,,,No,,Yes,No,Yes,,Stored Agency emails used for validation.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Intelligent Document Processing for Workflow Automation,Department of Homeland Security,DHS,ICE,Mission-Enabling,,None of the above.,"Business units within ICE leverage these services to automate repeatable, time-consuming processes such as invoice processing, and form entry validation and extraction. This platform will provide Optical Character Recognition and machine learning models to verify, extract, and classify information from ICE forms. Using AI to provide information extraction for these processes saves ICE personnel a significant amount of time while improving data quality and enabling automation. ","This platform will provide Optical Character Recognition and machine learning models to verify, extract, and classify information from ICE forms.",Operation and Maintenance,Neither,1/1/2019,1/1/2019,6/1/2019,,Developed with contracting resources.,70CTD021FR0000248; 70CDCR24FR0000048; 70CMSD21FR0000169; 70CTD021FR0000248; 70CTD023FR0000003; 70CTD020FR0000006; 70CTD022FR0000005,No,,,No,,Yes,No,Yes,,"ICE’s document understanding platforms provide out-of-the box models to verify and/or extract information from a variety of document types. These platforms also include the ability to create an ML feedback loop to tailor the models to try to improve the accuracy in extracting data fields from multiple document types. In this scenario, ICE would fine-tune the document understanding model using the documents submitted to the understanding workflow.","Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,"UiPath Suite, Azure AI Document Intelligence",More than 12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, "Cybersecurity Threat Management, Detection, and Response",Department of Homeland Security,DHS,ICE,Mission-Enabling,,None of the above.,"Machine Learning (ML) is used to analyze historical ICE cybersecurity data, create activity baselines, and detect anomalous behaviors for alerting. By integrating ML with other advanced analytics and correlation rules, AI identifies patterns indicative of security incidents, enabling SOC analysts to detect and respond to threats more quickly than previously possible.",The output of the analytics will be anomaly detection from patterns in security tool data correlated into alerts for SOC analysts.,Operation and Maintenance,Neither,5/21/2020,9/17/2020,3/16/2021,,Developed in-house.,,No,,,No,,No,Yes,No,,"N/A - The Agency does not own any of the data used to train, fine-tune, and/or evaluate performance of the model(s) used in this use case.","Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,No,No – agency does not have access to source code.,,Yes,"Xploitation Platform Leveraging Offensive Information Technology, Phishing Defense Solution, Audit Log Management Solution (ALMS), Endpoint Detection and Response, EMVEC",Less than 6 months,No,,No,,No,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Translation and Transcription for Investigative Data,Department of Homeland Security,DHS,ICE,Law & Justice,,None of the above.,"HSI investigators often encounter data from various sources, including legal and administrative processes, enforcement actions, and open-source materials, in languages other than English. To unlock the value of this data, it must be translated into English before further analysis can be conducted. The Translation and Transcription Service leverages neural machine translation (NMT) models for text translation and automatic speech recognition (ASR) and deep neural network (DNN) models with normalization for voice-to-text transcription. This innovative approach enables users to quickly triage large datasets and identify key information relevant to investigations. Any data deemed critical for court proceedings is then submitted to certified human translators for final review, ensuring that government resources are allocated efficiently and only used for necessary translations and transcriptions. ","The Translation and Transcription Service leverages neural machine translation (NMT) models for text translation and automatic speech recognition (ASR) and deep neural network (DNN) models with normalization for voice-to-text transcription. ",Operation and Maintenance,Neither,10/1/2022,10/1/2022,2/1/2024,,Developed with contracting resources.,"70OCTD22FR00000002, 70OCTD22FR00000235, 70OCTD22FR00000009",No,,,No,,Yes,Yes,Yes,,"AI models within the use case are not trained on agency data. Open-source models (i) Whisper is trained on 680K hours of multilingual and multitask supervised data collected from the web, (ii) No Language Left Behind (NLLB) is trained on a combination of publicly available datasets (additional information available in Section 5 of Meta’s NLLB whitepaper: https://research.facebook.com/file/585831413174038/No-Language-Left-Behind--Scaling-Human-Centered-Machine-Translation.pdf). ","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,"RAVEn, Azure AI Translator",Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Biometric Check-in for ATD-ISAP (SmartLINK),Department of Homeland Security,DHS,ICE,Law & Justice,,None of the above.,"ISAP Biometric Monitoring App is a technology option that allows participants to report in using a smartphone. This app verifies a participant’s identity, determine their location, and quickly collect status change information. The app adds functionality not available with telephonic and is less intrusive than a GPS unit. ISAP monitoring app limits in-person interactions of routine check-ins, allowing more time to be allocated to non-compliant participants, complex removal proceedings cases and docket management.","There are two outputs related to using ISAP Biometric Monitoring App. Either a participant “passes” (biometric match) or the photo is moved to a “pending review” status. In either scenario, a human can evaluate the response.",Operation and Maintenance,"Rights-Impacting ",2/1/2018,2/1/2018,2/1/2018,,Developed with contracting resources.,70CDCR20D00000011,No,,,No,,Yes,Yes,No,,"N/A - The Agency does not own any of the data used to train, fine-tune, and/or evaluate performance of the model(s) used in this use case.","Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,No,No – agency does not have access to source code.,,Yes,ISAP IV (SmartLink) IATSS,Less than 6 months,No,,No,,Other,N/A,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,Yes,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.","Environmental variables may negatively impact model performance. At enrollment, participants meet with a Case Manager, who explains the program in the participant’s preferred language. Participants receive guidance on the use of the system and environmental parameters which may affect results (e.g. poor lighting, busy backgrounds). All nonmatches are reviewed by human examiner and considered for a redo before the check in is officially marked as a nonmatch.  ",Yes – by the CAIO,"Automated and Regularly Scheduled Updates: Some aspects of the plan to monitor the AI system post-deployment are automated including re-training of models after detecting drift; however, data science teams are still significantly involved in the monitoring and re-deployment process.",No - Some individual decisions or actions require direct human oversight.,,In-person,,No,"There are two outputs related to using ISAP Biometric Monitoring App. Either a participant “passes” (biometric match) or the photo is moved to a “pending review” status. In either scenario, a human can evaluate the response.",None of the above,,No – it is not operationally practical to offer this.,"A human is required to evaluate all the ""pending review"" status items regardless of outcomes.",Yes, Email Analytics for Investigative Data,Department of Homeland Security,DHS,ICE,Law & Justice,,None of the above.,"HSI personnel encounter large volumes of legally acquired, multilingual email data that must be prepared (ingested, triaged, translated, searched and filtered) before it can be analyzed to support investigations. The Email Analytics application eliminates manual data preparation processes, and leverages machine learning to conduct spam message classification, translation, and entity extraction, including names, organizations, or locations. It also utilizes HSI's AI-enabled translation capabilities (see related use case “Translation and Transcription for Investigative Data”) for translation of emails in other languages to English. The output reduces time and resources spent preparing data, increases the analytic utility of the data, and allows HSI personnel to more quickly conduct analysis on the information.","The output is email data that has undergone spam message classification, translation, and entity extraction, including names, organizations, or locations.",Operation and Maintenance,Neither,7/1/2019,8/1/2021,8/1/2021,,Developed with contracting resources.,"70OCTD22FR00000002, 70OCTD22FR00000235, 70OCTD22FR00000009",No,,,No,,Yes,Yes,Yes,,Lawfully obtained email data.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Repository for Analytics in a Virtualized Environment,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Mobile Device Analytics for Investigative Data,Department of Homeland Security,DHS,ICE,Law & Justice,,None of the above.,"Mobile device analytics capabilities were developed to meet the growing need for investigators to quickly and effectively review and analyze the massive amounts of data extracted from mobile devices obtained in court-ordered seizures through a warrant or other investigative due process. The solution empowers investigators and analysts to identify and extract critical evidence, relationships, and networks from mobile device data, leveraging machine learning capabilities to determine locations of interest. Present capabilities are limited to the use of an algorithm for clustering of geo-location data, allowing investigators to identify 'stops' and 'overlaps' of locations from lawfully seized phones, where a 'stop' is a location a phone may have remained for a period of time and an 'overlap' is a location where multiple phones may have visited. By streamlining the analysis process, these capabilities aim to enhance the efficacy of HSI agents and analysts by providing advanced data processing to identify the most relevant information for lead generation in a shorter amount of time than manually processing, ultimately contributing to the disruption and dismantling of criminal networks and the protection of national security.","Present capabilities are limited to the use of an algorithm for clustering of geo-location data, allowing investigators to identify 'stops' and 'overlaps' of locations from lawfully seized phones, where a 'stop' is a location a phone may have remained for a period of time and an 'overlap' is a location where multiple phones may have visited.",Operation and Maintenance,Neither,6/1/2019,10/15/2021,9/1/2022,,Developed with contracting resources.,"70OCTD22FR00000002, 70OCTD22FR00000235, 70OCTD22FR00000009",No,,,No,,Yes,Yes,Yes,,"Lawfully obtained mobile device extraction data, including location data.","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Repository for Analytics in a Virtualized Environment,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Identification Card and Travel Document Code Detection,Department of Homeland Security,DHS,ICE,Law & Justice,,None of the above.,"The Identification Card and Travel Document Code Detection service leverages mobile phone cameras to quickly and accurately read 2D Data Matrix Codes on US Driver's Licenses and ID cards, as well as Machine Readable Zones (MRZ) on travel documents like Passports and Passport cards. This innovative solution utilizes machine learning to enhance the detection of these codes, allowing for seamless scanning and automatic population of detected information into corresponding text fields within HSI mobile apps. By streamlining the code recognition process, the service significantly reduces the time required for users to read cards or documents, eliminating the need for manual data entry and resulting in more efficient, in-person interactions with individuals.","The output is a dataset of structured identification data. ",Operation and Maintenance,Neither,6/1/2020,11/1/2020,6/1/2021,,Developed with contracting resources.,"70OCTD22FR00000002, 70OCTD22FR00000235, 70OCTD22FR00000009",No,,,No,,Yes,Yes,Yes,,"N/A - The Agency does not own any of the data used to train, fine-tune, and/or evaluate performance of the model(s) used in this use case.","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Repository for Analytics in a Virtualized Environment,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Normalization Services,Department of Homeland Security,DHS,ICE,Law & Justice,,None of the above.,"HSI utilizes artificial intelligence to enhance data accuracy and efficiency by verifying, validating, correcting, and normalizing various types of information, including addresses, phone numbers, names, and ID numbers. This process helps to eliminate data entry errors, detect intentional misidentification, and connect related information across multiple datasets, ultimately reducing the time and resources required for investigations. The machine learning-powered normalization services offered by HSI include converting ambiguous addresses into usable formats, identifying ID types from partial information, categorizing names with complex suffixes and family names, and standardizing phone numbers to the E164 format, including determining their originating county. By normalizing and improving the quality of investigative datasets, HSI is able to use more advanced tools to find correlations and leads that would have otherwise gone undetected without extensive manual effort.","The output includes normalized data that improves search capability during investigations. This includes normalizing data to update less well-defined addresses into usable addresses for analysis- (such as those using mile markers instead of a street number); inferring ID type based on user-provided ID value (such as distinguishing a SSN from a DL number without additional context); categorizing name parts while taking into account additional factors (including generational suffixes and multi-part family names); and validating and normalizing phone numbers to the E164 standard, including their identified county of origin.",Operation and Maintenance,Neither,6/1/2019,10/1/2019,4/1/2021,,Developed with contracting resources.,"70OCTD22FR00000002, 70OCTD22FR00000235, 70OCTD22FR00000009",No,,,No,,Yes,Yes,Yes,,"Data holdings within HSI case files that require normalization (e.g. subpoenaed phone record). This includes, but is not limited to, evidentiary records containing phone numbers, names, addresses, and ID numbers.","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Repository for Analytics in a Virtualized Environment,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, JES and Appropriations Insight,Department of Homeland Security,DHS,MGMT,Mission-Enabling,,None of the above.,"Purpose: Converts scanned financial tables from PDFs into structured, machine-readable data while maintaining multi-year spending relationships. Benefits: Eliminates manual data entry, reduces errors, and significantly speeds up the process of consolidating historical financial data from legacy documents.",Structured tables in place of free text to provide a machine-readable dataset.,Acquisition and/or Development,Neither,3/1/2024,4/1/2024,,,Developed with contracting resources.,70RDAD22FR0000064,No,,,,,,,Other,"OCFO uses Collibra, through CDOD's Matrix environment. The catalog build-out is WIP.","N/A - The Agency does not own any of the data used to train, fine-tune, and/or evaluate performance of the model(s) used in this use case.","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,,,,No,,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, CFO Companion,Department of Homeland Security,DHS,MGMT,Mission-Enabling,,None of the above.,"Provides authorized staff with an intuitive, conversational interface to query and analyze DHS CFO financial data and reports. Benefits: Democratizes access to financial information, reduces time spent searching through documents, and enables quick self-service analytics without specialized database knowledge.",On-demand information retrieval via natural language processing.,Acquisition and/or Development,Neither,4/1/2024,4/8/2024,,,Developed with contracting resources.,70RDAD22FR0000064,No,,,,,,,Other,"OCFO uses Collibra, through CDOD's Matrix environment. The catalog build-out is WIP","Internal system and process information, such as the link to a visualization, or data attribute information tied to publicly available finanacial data.","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,,,,Yes,CFO Horizon,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Climate Change Assessment Tool,Department of Homeland Security,DHS,MGMT,Mission-Enabling,,None of the above.,"Our Nationis already experiencing the adverse impacts from climate change, from sea-level rise, extreme heat, flooding, and drought, to changes in migration patterns and harmful effects on workforce health. DHS needs to be able to better understand and predict where extreme weather conditions will affect DHS assets, including our facilities and our workforce, in the near and long-term. In support of the DHS Climate Action Plan, this AI pilot will enable the detection and streamliinng the assessment of cost/benefit data driven decisions like the location of facilities, prioritization of O&M projects and the health and safety of the DHS workforce. Benefits include cost savings, time savings and improved access to critical data and ability to make decisions, improved health and safety of the workforce.",Outputs include the prediction and early detection of extreme weather risks and an integrated understanding of the affects this will have on critical DHS resources. A secondary output is cost-benefit recommendations.,Acquisition and/or Development,Neither,3/1/2024,11/19/2024,,,Developed with both contracting and in-house resources.,70RDAD23FR0000148,No,,,,,,,Yes,,"DHS Real Property/Facility Data, DHS Employee and Contractor Data, DHS Facility utilization (e.g. energy usage) Data","Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",None,,,,,Yes,Management Cube (MGMT Cube),More than 12 months,Yes,,No,,Other,N/A,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Spending Analysis and Budget Execution Risk (SABER) Model,Department of Homeland Security,DHS,MGMT,Mission-Enabling,,None of the above.,"Purpose: Predicts potential budget execution issues by analyzing historical spending patterns across various Treasury accounts and classifications. Benefits: Early identification of spending anomalies allows proactive budget management and reduces the risk of under/overspending.","Warnings, flags for review and comparison via prediction and classification.",Acquisition and/or Development,Neither,1/22/2024,2/19/2024,,,Developed with contracting resources.,70RDAD22FR0000064,No,,,,,,,Other,"OCFO uses Collibra, through CDOD's Matrix environment. The catalog build-out is WIP",Treasury data with slight report-centric fields added. The data is publicly available.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,,,,No,,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, AdaptiveMFA,Department of Homeland Security,DHS,MGMT,Mission-Enabling,,None of the above.,"Adaptive Multi-factor Authentication (AMFA) introduces additional intelligence into Identity flows by taking into account the authentication context data during the authentication. Using the data, DHS is able to adapt security and authentication policies to enhance the security to DHS systems.","The input includes: device, network, location, travel, IP, and external data from third parties and endpoint security integrations. The output are Risk ratings HIGH, MEDIUM, LOW for each authentication attempt which can be configigured to require stricted access comtrol policies.",Acquisition and/or Development,Neither,3/1/2024,3/30/2024,,,Developed with contracting resources.,47QFRA24F0005,No,,,,,,,No,,The system collects signal data during authentication to dynamically build behaviors of the user.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,,,,Yes,The DHS Next Generation Authentication as a Service (DHSAuthPortal) System,More than 12 months,Yes,,No,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, ESEC Inquiry (STORM) Summarization,Department of Homeland Security,DHS,MGMT,Mission-Enabling,,None of the above.,"ESEC-STORM AI employs Generative Artificial Intelligence (Gen-AI) technology to automate the creation of document summaries. This advanced system facilitates the automatic integration of these summaries into the System of Tracking, Operations, and Record Management (STORM), thereby optimizing the management of correspondence and information requests. ","When a user creates a new work package and uploads a letter, it activates a Power Apps workflow that creates a summary.",Acquisition and/or Development,Neither,3/1/2024,4/1/2024,12/13/2024,,Developed in-house.,,No,,,No,,No,No,Yes,,"N/A - The Agency does not own any of the data used to train, fine-tune, and/or evaluate performance of the model(s) used in this use case.","Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Dynamics Online As a Service; BIaaS,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, One Health Threat Detection and Risk Assessment Platform (OH-TREADS) / Planner,Department of Homeland Security,DHS,OHS,Health & Medical,,None of the above.,"Visuals will be created using a broad variety of data pulled from open-source, public, and non-public sources. AI/ML will make it possible to ingest data from a variety of digital formats, and translate it to usuable information into machine-readable data. The volume of data, and lack of clear data collection standards, requires AI to help merge streams with different data ontologies while facilitating data interoperability with mis-paired data sets. Neither activity is, or can be, practically performed by people. AI/ML will be further used in Planner to generate risk and predictive scores in addtion to displaying relevant information and analyses that help analysts understand health security threats and broadly monitor the health security landscape. AI/ML is meant to provide comprehensive situational awareness for health surveillance and public health response, with target capabilities that: enable global situational awareness of current and potential health risks from a One Health perspective; assist in early warning of health threats by location, facility, and species; aid in rapid identification of health threats at population, facility, and greater geographic resolution; and support data-driven decision making to prevent, mitigate, and respond to health threats.","The main AI system outputs will be anomaly detection following the translation of information into a structured, machine-readable datasets. AI/ML is then used for risk identification and disease prediction that are overlayed on a map and displayed with other wholistic visuals. The visuals and analytics can then be combined with critical infrastructure locations, resources, or capabilities (feeral, state, local, tribal, and territorial) to aid response and decision making.",Initiated,Neither,8/1/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Sentiment Analysis and Topic Modeling (SenTop),Department of Homeland Security,DHS,MGMT,,,None of the above.,"The initial purpose of the Sentiment Analysis and Topic Modeling (SenTop) project was to analyze survey responses for DHS’s Office of the Chief Procurement Officer related to contracting. However, it has evolved to be a general-purpose text analytics solution that can be applied to any domain or area. It has also been tested and used for HR topics. SenTop is a DHS-developed Python package for performing descriptive text analytics, specifically, sentiment analysis and topic modeling on free-form, unstructured text. SenTop uses several methods for analyzing text, including combining sentiment analyses and topic modeling into a single capability, permitting identification of sentiments per topic and topics per sentiment. Other innovations include the use of polarity and emotion detection, fully automated topic modeling, and multi-model/multi-configuration analyses for automatic model/configuration selection. The code has been established, performs an analysis, and provides a report, but it is only accessed and run by one person per customer request.","This use case is inactive. Please see the ""intended purpose and expected benefits"" for a full description of this use case. ",Retired,,,,,10/1/2024,Developed in-house.,,No,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, DHS-Chat,Department of Homeland Security,DHS,MGMT,Mission-Enabling,,None of the above.,"This is a chatbot based on a Large Language Models (LLM) for internal DHS employee use - It's like chat-GPT for DHS but approved for use with non-classified but internal information (this includes FOUO (For Official Use Only) CUI (Controlled Unclassified Information) due to its improved security compared to publicly available chatbots. This tool is able to dynamically create written content through text prompts submitted by the user. Approved applications of this tool to DHS business include generating first drafts of documents that a human would subsequently review, conducting and synthesizing research on open-source information and internal documents, and developing briefing materials or preparing for meetings and events.",The internally available generative AI tool outputs text based on the users input.,Operation and Maintenance,Neither,10/1/2024,10/1/2024,12/12/2024,,Developed in-house.,,No,,,No,,No,Yes,No,,"N/A - The Agency does not own any of the data used to train, fine-tune, and/or evaluate performance of the model(s) used in this use case.","Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",None,,Yes,"Yes – agency has access to source code, but it is not public.",,No,(Pending),Less than 6 months,Yes,,Other,Compute resourcing increases are managed through the centralized IT infrastructure and working with Microsoft Azure.,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, User and Entity Behavior Analytics (UEBA),Department of Homeland Security,DHS,MGMT,Mission-Enabling,,None of the above.,"Enhanced Threat Detection: Identify patterns in user behaviors that deviate from normal baselines, signaling potential insider threats or security risks. Continuous Risk Assessment: Move from static vetting to a continuous vetting process by monitoring real-time activities and interactions with secure information. Improved Incident Response: Enable rapid responses to high-risk behaviors, escalating alerts for timely intervention by security personnel."" ","Behavioral Baseline Modeling: Develop a baseline for each individual based on regular access patterns, network usage, and interactions with classified data or secure areas. Anomaly Detection: Employ machine learning models to detect deviations from established baselines, such as unusual access times, atypical access to high-sensitivity resources, or excessive data downloads. Risk Scoring: Assign a risk score to each user based on observed anomalies, factoring in historical behavior, job role, and access level, allowing security teams to prioritize investigations. Automated Alerts & Reporting: Generate automated alerts for high-risk behaviors or patterns of concern and deliver timely reports to personnel security teams for further investigation."" ",Operation and Maintenance,Neither,10/1/2020,4/3/2021,7/8/2023,,Developed with both contracting and in-house resources.,"PIID Pending, not yet issued.",No,,,No,,Yes,Yes,Yes,,"PII, SPII , Clearance and Background Investigation Data: Data from security clearances, background investigations, and adjudication records.","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Vetting Identities for an Enterprise Workforce,More than 12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Text Analytics for Survey Responses (TASR),Department of Homeland Security,DHS,MGMT,Mission-Enabling,,None of the above.,"The intended purpose of the AI is to perform topic modeling, sentiment analysis, or other text classification tasks on responses provided to internal staff DHS Pulse Survey questions. Text Analytics for Survey Responses (TASR) is an application for performing Natural Language Processing (NLP) and text analytics on survey responses. It is currently being applied by DHS Office of the Chief Human Capital Officer (OCHCO) to analyze and extract significant topics/themes from unstructured text responses to open-ended questions in the quarterly DHS Pulse Surveys. Results of extracted topics/themes are provided to DHS Leadership to better inform agency-wide efforts to meet employees’ basic needs and improve job satisfaction","The systems outputs include a set of topics inferred or surfaced from the raw text comment data, as well as sentiments or other classifications inferred from the data.",Operation and Maintenance,Neither,10/1/2022,10/1/2022,11/1/2022,,Developed in-house.,,No,,,No,,No,Yes,No,,"N/A - The Agency does not own any of the data used to train, fine-tune, and/or evaluate performance of the model(s) used in this use case.","Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",None,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,No,,No,,Other,N/A,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, MiX Phenotyping,Department of Homeland Security,DHS,OHS,Health & Medical,,None of the above.,"Electronic medical records can provide key data for health security insights. AI/ML makes it possible to translate these records into machine-readable data, which is the first step to finding these health security insights. AI/ML will then be used to find clinical patterns across the data, such as patterns in symptoms over time and location. AI/ML will also be used to automate the process for detecting new trends (anomalies) in these clinical patterns. These health security insights can alert about potential threats, inform messaging, and provide decision support to medical and public health partners.","The main AI system outputs will include anomaly detection for emerging trends in clinical record patterns. AI/ML outputs will also be used to find these clinical patterns by clustering, classification and topic modeling. These clinical patterns will finally be output as linear reference models that are simple enough to be interpreted and guided by human clinicians using human-in-the-loop collaboration.",Initiated,Neither,1/1/2021,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, MiX Indicators,Department of Homeland Security,DHS,OHS,Health & Medical,,None of the above.,"Online news and other open-source media can be used to quickly detect and respond to emerging health security threats. However, it is not practical for people to read and scan all online news stories every day for key terms. AI/ML makes it possible to read, digest and organize news stories to look for key threat terms. AI/ML will also be used to predict normal trends in news stories. This makes it possible to detect when an unusual number of health security threat terms are found in the news. This use of AI/ML helps prepare for, respond to, and protect from potential health security threats.","The main AI system output will be anomaly detection which represents two elements: 1) AI/ML predictions for usual trends in news story key terms vs. 2) the actual daily number of mentions for key terms in the news. When the actual number of mentions for key terms exceeds modeled predictions, these will be detected as anomalies.",Initiated,Neither,1/1/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, MiX MedINT (Medical Intelligence Dashboard and Canvas),Department of Homeland Security,DHS,OHS,Health & Medical,,None of the above.,"Health security visuals will be created using a broad variety of data pulled from online sources. AI/ML will make it possible to translate this information into machine-readable data. AI/ML will also find relevant terms in these online articles, such as for locations and diseases of interest, label articles accordingly, and rank articles for relevance to topics of interest. These necessary steps cannot practically be performed by people. AI/ML will then support MedINT Canvas to visualize connections between topics of interest, such as the presence of key symptoms and disease diagnoses in certain locations. Given the wide variety of potential health security threats, wholistic visual tools are needed to allow analysts to broadly monitor the health security landscape.","The main AI system output will be the translation of online information into a structured, machine-readable dataset. This includes language translation for non-English articles, labeling articles by health threat types, and ranking articles by relevance to health threats. This dataset will then be used to create the wholistic visuals in Canvas and Dashboard.",Initiated,Neither,1/1/2021,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Low Probability of False Alarm (Low-Pfa) Algorithm for on-person screening.,Department of Homeland Security,DHS,TSA,Transportation,,None of the above.,"The purpose is to reduce alarm rates while providing gender neutral detection which would increase passenger throughput and experience. Utilizes Machine Learning (ML) to improve detection performance while decreasing alarm rates and passengers touch rates. The algorithm is gender agnostic which no longer requires officers to select a passengers gender prior to being scanned. AIT throughput and AIT utilization have increased with this new algorithm. Note: Once the algorithm is trained, it is locked down and no longer learning.",The AI outputs target coordinates to the operator viewing station which is viewed as a bounding box on a representative human figure.,Implementation and Assessment,Both,9/1/2021,9/1/2022,,,Developed with contracting resources.,Contract: HSTS0417DCT1087 Task Order: 70T04022F7672N011,Yes,Transportation Security Administration – DHS ,Completing TSA passenger screening,No,,No,No,No,,"N/A - The Agency does not own any of the data used to train, fine-tune, and/or evaluate performance of the model(s) used in this use case.","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,More than 12 months,No,,No,,Other,N/A,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,Yes,"Impact evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution and has utilized randomized experiments with a control group or other counterfactual, social systems analysis, or other rigorous research methodologies to evaluate impact and identify potential harm to users as well as broader groups of people.","Security risks include false negatives allowing threats to get through the sterile side of the airport or high false alarm rates slowing operations. The program follows the Test & Evaluation (T&E) process documented in the TSA T&E Policy, T&E Guidebook, TSA Acquisition Qualification Policy, and TSA Third Party Testing Strategy, which includes independent test & evaluation. Various types of testing were conducted, including functional testing, developmental test & evaluation, independent test & evaluation, and integration/implementation testing. Tests were conducted on a variety of body types with varying concealment locations, concealment methods, and target types. Metrics include probability of detection, probability of false alarm, mean time between failure, mean down time, mean time to repair, and percentage of eligible passengers unable to be screened. All tests met or exceeded expectations. TSA performs regular checks to verify performance of the scanner (which includes the AI model) using Operational Test Kits (OTK). OTKs are used daily, whenever a scanner is moved, whenever a scanner is power-cycled, and as needed to verify performance. OTKs are created by each OEM and are specific to their scanners. They contain items that when concealed on a scanned person will result in a positive scan for a target. ",Yes – by the CAIO,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,Online - in the terms or instructions for the service.,,No,"The program follows the Test & Evaluation process documented in the TSA T&E Policy, T&E Guidebook, TSA Acquisition Qualification Policy, and TSA Third Party Testing Strategy, which includes independent test & evaluation.","Direct user testing,General solicitations of comments from the public",,Yes,,Yes, Automated Target Recognition (ATR) Developments for Standard Screening,Department of Homeland Security,DHS,TSA,Transportation,,None of the above.,"The purpose of this use case is to improve upon Automated Target Recognition (ATR) algorithms used to reduce privacy concerns because a TSO is no longer required to view Advanced Imaging Technology (AIT) images. The expected benefits are to increase detection, reduce false alarms, and improve efficiency and passenger experience.","The system reproduces the threat location, which is viewed as a bounding box on a representative human figure, for TSO resolution.",Acquisition and/or Development,Both,6/22/2017,6/22/2017,,,Developed with contracting resources.,70T04023C7573N001,Yes,Transportation Security Administration – DHS ,Completing TSA passenger screening,,,,,No,,TSA owns AIT image scans which are used for algorithm development. The scans contain threat images and are potentially sensitive.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,,,,No,,More than 12 months,No,,No,,Other,N/A,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Accessible Property Screening (APS) Checkpoint CT Prohibited Items (PI) Detection,Department of Homeland Security,DHS,TSA,Transportation,,None of the above.,"This AI helps airplane luggage checks by the TSA officers scanning bags to have a continuous always watching partner to alert them to anything suspicious. Currently, a Transportation Security Officer (TSO) who is assigned to every X-ray equipment at an airport checkpoint visually inspects each image. This officer resolves the system generated explosive alarms as well as visually inspecting the image for the presence of non-explosive prohibited items such as guns and sharp objects (see TSA Travel site). TSA is working on developing new Artificial Intelligence/Machine Learning (AI/ML) algorithms to automate the search for the non-explosive prohibited items (e.g. guns, knives, etc.). Once a threat is found, the algorithm displays bounding boxes around the suspect item for the operator to then investigate and adjudicate. These AI solutions benefit the public by providing a consistent and uninterrupted level of threat detection as an added layer of security. The ML algorithms allow the TSA officers to be more flexible and to better prioritize their attention on important items to improve security.",AI system output is a set of 3-dimentional bounding boxes that is displayed on the X-ray image. The bounding boxes are placed on top of objects or areas where the algorithm believes it has found a prohibited item (threat object).,Acquisition and/or Development,Both,10/1/2019,10/1/2019,,,Developed with contracting resources.,70T04024F7573N010 - Smiths; 70T04023C7573N003 - SureScan (IDSS); 70T04023D7672N007 - Analogic,Yes,Transportation Security Administration – DHS ,Completing TSA passenger screening,,,,,No,,"CT images of government scanned bags with threats, additionally have Stream of Commerce CT images collected at several airports. The images are in DICOS or vendor proprietary image format.","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,,,,Yes,CheckPoint Property Screening System (CPSS),Less than 6 months,No,,No,,Other,N/A,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Walk-Through Metal Detector (WTMD) Alternative Automated Target Recognition (ATR) Developments,Department of Homeland Security,DHS,TSA,Transportation,,None of the above.,Artificial intelligence (AI)-enhanced Millimeter wave (mmWave) detectors are used as an alternative to Walk-Through Metal Detectors (WTMDs) for passenger screening to detect both metallic and non-metallic threats and prohibited items on passengers at the security checkpoint. These detectors are an improvement over traditional walk-through metal detectors and provide increased security and a better passenger experience.,The AI outputs target coordinates to the operator viewing station which is viewed as a bounding box on a representative human figure.,Acquisition and/or Development,Both,9/30/2021,9/30/2021,,,Developed with contracting resources.,70T04024C7573N002,Yes,Transportation Security Administration – DHS ,Completing TSA passenger screening,,,,,No,,TSA owns AIT image scans which are used for algorithm development. The scans contain threat images and are potentially sensitive.,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,,,,No,,More than 12 months,No,,Other,Not yet provisioned.,Other,Not yet provisioned.,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Conversation Training and Feedback Simulator,Department of Homeland Security,DHS,TSA,Education & Workforce,,None of the above.,"TSA's Training and Development office is using an off-the-shelf, on-demand, employee learning/training platform which contains commercial training on many topics tailored to each user's interests and skill level to support their personalized training plans and goals. One available component of the platform uses a large language model (LLM) as part of an interactive, dialogue-style training functionality to communicate with and advise the learner to develop critical non-security sensitive soft skills such as customer service and communicating on difficult topics.",Text responses to questions or answers to a question,Acquisition and/or Development,Neither,6/1/2024,9/1/2024,,,Developed with contracting resources.,70RDAD23FR0000044,No,,,,,,,No,,None,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",Sex/Gender,,,,,Yes,Percipio,Less than 6 months,No,,No,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Answer Engine,Department of Homeland Security,DHS,TSA,Mission-Enabling,,None of the above.,"TSA aims to enhance its capabilities in managing and analyzing complex data, ultimately contributing to more effective and efficient security operations and optimizing the TSA's operational workflows and support capabilities.","This platform is anticipated to harness the power of AI to provide intelligent, context-aware responses and insights.",Acquisition and/or Development,Neither,10/1/2024,10/1/2024,,,Developed with contracting resources.,70T03024F7667N086,No,,,,,,,No,,SOP and Policy documents for conducting TSO duties.,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,,,,No,,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Automated Field Data Collection,Department of Homeland Security,DHS,TSA,Transportation,,None of the above.,"The AI will analyze screening environments via CCTV footage and extract passenger processing times of various steps within the screening processes. Enabling AI to extract and visualize this data will enable TSA to make data informed decisions while testing or deploying new screening equipment, identify anomalies, establish real-world rates and standards, and reduce or eliminate TSA’s need to deploy data collection teams, resulting in real-time data collection and significantly reduced computational time of findings. ","AI system outputs multiple decisions to include screening location performance, rates and standards of the end-to-end screening system, and passenger wait times. ",Acquisition and/or Development,Neither,4/1/2023,9/1/2024,,,Developed with both contracting and in-house resources.,70T04024CSOP7573N001,Yes,Transportation Security Administration – DHS ,Completing TSA passenger screening,,,,,No,,Screening process rates & standards.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,,,,No,,6-12 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, Plan of Day Staff Optimization,Department of Homeland Security,DHS,TSA,Transportation,,None of the above.,"Plan of Day will automate TSA screening staff optimization. ","Staffing operations models prescribing when screening lanes should be opened/closed, when/where screening staff is required to absorb operational peaks, determining optimal gender and certification ratios, recommending when to schedule overtime/shift adjustments, drafting lane rotation plans, and informing national TSA staffing requirements as prescribed optimization plans deviate as airline schedules shift. ",Acquisition and/or Development,Neither,10/1/2017,10/1/2017,,,Developed with both contracting and in-house resources.,"DHS, S&T Contract Number: 70RSAT21PI00001",Yes,Transportation Security Administration – DHS ,Completing TSA passenger screening,,,,,Yes,,"STIP CAT data and PMIS passenger throughput data; PIMS historic throughput of screening equipment; Master Checkpoint List/Location Codes; Salesforce Master Airport List; HR data via eTAS, NFC, and OLC; AIMs data for airport logistic information; OAG for flight information; Google weather and traffic ","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,,,,No,,More than 12 months,Other,DHS Science and Technology accepted the project via a formal intake process.,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, TSA Contact Center Virtual Assistant,Department of Homeland Security,DHS,TSA,Government Services (includes Benefits and Service Delivery),,None of the above.,"The three goals for TSA’s virtual assistant are: 1) Improve Ease / Efficiency – deliver timely, accurate responses to customers within / outside of TCC operational hours; 2) Improve Effectiveness – collect new customer insights to help drive agency improvements; 3) Fiscal Responsibility – leverage automation to meet increasing demand without the need for additional resources.","TSA’s virtual assistant, as a Predictive AI tool, will not be able to create and provide original content to customers. It will be limited to providing responses to customer inquiries based on the existing content in the TCC’s knowledge library. However, it will record transactional data related to these customer inquiries (e.g., customer inputs, assistant outputs, topic classifications, etc.). TSA’s virtual assistant will create transactional data regarding customer inquiries in a consistent manner to our existing email and phone channels. TSA uses this transactional data to guide customer experience improvement efforts. This data offers TSA insights into: what is working well, where the pain points are, where to improve public information, and what services are most requested.",Acquisition and/or Development,Neither,1/10/2024,8/1/2024,,,Developed with both contracting and in-house resources.,70T01024F5600N002,Yes,Transportation Security Administration – DHS ,Getting help from TSA,,,,,Yes,,"TSA.gov FAQs (publicly available), TSA Contact Center (TCC) inquiry data (e.g. why customers contacted us; how we responded to them)(not pubicly available), TCC knowledge library (i.e. approved response language; currently used by TCC agents when responding to customers via email and phone channels)(not publicly available; but derived from TSA.gov FAQs).","Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,,,,Yes,Customer Service Center - Salesforce,Less than 6 months,Yes,,No,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Machine Learning Analysis Applied to Cyber Threat Hunt Data,Department of Homeland Security,DHS,TSA,Mission-Enabling,,None of the above.,"Cyber threat hunts typically involve a vast amount of data. Machine learning models can quickly and efficiently process this data as well as more effectively identify anomalous activity than humans. This could improve the efficiency and quality of cyber threat hunts by detecting suspicious behavior more quickly and increasing the amount of data that can be analyzed during a hunt.","Currently it is a list of potential anomalies or outliers within the system, but development is still ongoing.",Acquisition and/or Development,Neither,1/1/2024,2/1/2024,,,Developed with contracting resources.,GS-35F-525GA,No,,,,,,,No,,Data collected from TSA-owned assets through cyber threat hunt operations.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,,,,Yes,System name is operationally sensitive.,Less than 6 months,No,,No,,Other,N/A,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Synthetic Data for Improved Automated Threat Recognition (ATR) in Checkpoint Screening,Department of Homeland Security,DHS,TSA,Transportation,,None of the above.,"To create synthetic data that can be used to improve Automated Threat Recognition (ATR) algorithm development. Synthetic data can be quicker to produce which will improve effectiveness by addressing and adapting to new threats quicker. Accessible Property Screening (APS) and On-Person Screening (OPS) are working with vendors and evaluating AI-based synthetic data generation techniques to bolster the pool of training data available to develop machine learning algorithms in ATR applications.",Images that mimic the human body and various threats.,Initiated,Both,1/11/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, OTA Automated Passenger Screening Gate System,Department of Homeland Security,DHS,TSA,Transportation,,None of the above.,The purpose is to manage passenger flow while reducing human interaction by using AI to assess body positioning and automatically initiate the screening process when the passenger is in the optimum position.,"The AI system initiates an AIT scan when the passenger is in the optimum position. Depending on the results of the AIT scan, the passenger would be routed either for additional screening or to the re-composure area to claim their accessible property and transit into the sterile area.",Initiated,Neither,8/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Axon Capture Application,Department of Homeland Security,DHS,TSA,Law & Justice,,None of the above.,"The Transportation Security Administration (TSA) uses body-worn cameras that incorporate artificial intelligence (AI) technologies as a part of the underlying software which transcribes and translates video footage. The technology provides rapid access to LEO and Investigative data, through transcription. This will cut investigative processing time significantly, while also increasing accuracy.",The AI will transcript audio/video data and provide a printable artifact. The AI only provides recommendations the final usable data is reviewed and certified by TSA staff. ,Initiated,"Rights-Impacting ",10/1/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Contract Requirement Automation,Department of Homeland Security,DHS,TSA,Mission-Enabling,,None of the above.,"Eliminates the manual burden and reduces errors in document creation, where staff previously spent excessive time writing requirements to meet specific procurement needs from scratch. Increases documentation accuracy, consistency and standardization. Improves management and verification of requirement documents. Cost savings represent another major benefit, as the platform reduces labor costs associated with manual document creation and management. By streamlining the procurement process, the agency can complete more procurement actions with existing resources, maximizing taxpayer dollars. The automated tool also reduces the time spent on repetitive tasks, allowing for better resource allocation.","The tool primarily outputs automated document generation and recommendations for procurement documentation. The platform provides contextual recommendations during the document creation process to ensure completeness and compliance with procurement requirements. The tool provides recommendations and decision support to guide users through the proper documentation structure and content requirements. The system's outputs always require human verification as part of the workflow, ensuring that all generated content is reviewed and approved by qualified personnel before being finalized in the procurement process.",Initiated,Neither,4/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, TSA Case Handling Platform,Department of Homeland Security,DHS,TSA,Mission-Enabling,,None of the above.,"The tool saves case workers manual time to download reports, order, and compile. The case workers can then use their time to work on cases rather than administrative tasks. Newton POC has the potential of streamlining collection processes related to cases, and creating custom reports from various materials during the case managers interview process, producing a centralized tool to manage and control all steps within each case.","The LLM automation capabilities include compiling, ordering, and exporting a PDF document that contains 6 key documents from the initial steps of formal complaint process, and 3 key documents at the final stages of the complaint.",Initiated,Neither,7/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, CDC Airport Hotspot Throughput (PageRank),Department of Homeland Security,DHS,TSA,,,None of the above.,"TSA launched the “Stay Healthy. Stay Secure.” campaign, which details proactive and protective measures implemented at security checkpoints to make the screening process safer for passengers and our workforce by reducing the potential of exposure to the coronavirus. The campaign includes guidance and resources to help passengers prepare for the security screening process in the COVID environment. A big part of that campaign was the development of the Centers for Disease Control and Prevention's Airport Hotspot Throughput. This capability determines the domestic airports that have the highest rank of connecting flights during the holiday travel season to help mitigate the spread of COVID-19. This capability is a DHS-developed artificial intelligence model written in Spark/Scala that takes historical non-PII travel data and computes the highest-ranking airports based on the PageRank algorithm. TSA does not make decisions about flight cancellations or airport closures. These decisions are made locally, on a case-by-case basis, by individual airlines, airports, and public health officials. TSA will continuously evaluate and adapt procedures and policies to keep the public and our workforce safe as we learn more about this devastating disease and how it spreads.","This use case is inactive. Please see the ""intended purpose and expected benefits"" for a full description of this use case. ",Retired,,,,,10/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Airport Throughput Predictive Model,Department of Homeland Security,DHS,TSA,Mission-Enabling,,None of the above.,This project was to create a predictive model for the passenger volume using the Security Operations throughput count from checkpoints to help with airport staffing.,"Once a month the data is ingested, the predictive model is trained, and predictions of airport checkpoint throughput are made for the airports.",Operation and Maintenance,Neither,6/1/2023,6/1/2023,4/1/2024,,Developed with both contracting and in-house resources.,"Performance Engineering Analytics – Contract#:47QRAD20D1175, Task Order:70T02023F7500N001",No,,,No,,No,Yes,No,,Secure Flight Passenger Data: passenger and airline reservation information received from airlines; PMIS Data: Secure checkpoint throughput counts by airport and checkpoint,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Secure Flight,More than 12 months,No,,Other,Performance Engineering has an on-prem environment with machine learning/modeling capabilities,Other,N/A - On-Prem,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Credential Authentication Technology with Camera System (CAT-2) and AutoCAT (CAT-2 in an e-gate form factor),Department of Homeland Security,DHS,TSA,Transportation,,None of the above.,"The Transportation Security Administration (TSA) uses AI-based, one-to-one (1:1) facial matching technologies at some checkpoints to assist human reviewers with traveler identity verification. The purpose and expected benefits of the technology include increased speed and accuracy of identity verification at the checkpoint while improving detection of imposters.","The system produces an recommendation to the Transportation Security Officer (TSO) to indicate if person presenting the identity document is similar to the face on the photo ID document. In the event of a non-match, the TSO is responsible for additional identity verification steps to verify the identity of the traveler.",Operation and Maintenance,"Rights-Impacting ",1/1/2022,7/1/2023,9/1/2023,,Developed with contracting resources.,70T04023D7672N0001,Yes,Transportation Security Administration – DHS ,Completing TSA passenger screening,No,,Yes,Yes,No,,"N/A - The Agency does not own any of the data used to train, fine-tune, and/or evaluate performance of the model(s) used in this use case.","Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",None,,No,No – agency does not have access to source code.,,Yes,STIP/CAT,6-12 months,No,,No,,Other,N/A,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,Yes,"Impact evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution and has utilized randomized experiments with a control group or other counterfactual, social systems analysis, or other rigorous research methodologies to evaluate impact and identify potential harm to users as well as broader groups of people.","In the event of a non-match, the traveler may make a second attempt or TSA may perform additional identity verification steps to verify the identity of the traveler. This process may add between 20 seconds or a few minutes to the identity verification and security screening process. This is mitigated through T&E and continuous monitoring.  ",Yes – by the CAIO,"Automated and Regularly Scheduled Updates: Some aspects of the plan to monitor the AI system post-deployment are automated including re-training of models after detecting drift; however, data science teams are still significantly involved in the monitoring and re-deployment process.",No - Some individual decisions or actions require direct human oversight.,,"Online - in the terms or instructions for the service.,In-person,Other",TSA Website,No,Test and Evaluation and continuous monitoring.,Direct user testing,,Yes,,Yes, PreCheck Touchless Identity Solution (TIS),Department of Homeland Security,DHS,TSA,Transportation,,None of the above.,"TSA is using Facial Comparison to verify a passenger’s identity at its security checkpoints and CMAP locations using the CBP Traveler Verification Service (TVS). This process streamlines passenger and crewmember identity verification, increasing the speed of security checks while maintaining a high degree of safety for all passengers and crewmembers.""","TSA is leveraging CBP's TVS system technology as an optional process for passengers or crew members traveling via certain airports who wish to further expedite their TSA PreCheck or crew member ID verification process. This additional TSA PreCheck feature is voluntary, and passengers may opt-out of the process at any time and instead choose the standard identity verification by a Transportation Security Officer (TSO). Crew members that wish to opt-out will be sent to the security checkpoint to process through screening.",Operation and Maintenance,"Rights-Impacting ",4/1/2016,4/1/2017,10/1/2018,,Developed with contracting resources.,024-000005265,Yes,Transportation Security Administration – DHS ,Completing TSA passenger screening,No,,Yes,Yes,Other,Data repository owned by another agency.,"N/A - The Agency does not own any of the data used to train, fine-tune, and/or evaluate performance of the model(s) used in this use case.","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,CBP TVS,Less than 6 months,Other,Provisioned through CBP.,Other,CBP has process in place to request access,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,Yes,"Impact evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution and has utilized randomized experiments with a control group or other counterfactual, social systems analysis, or other rigorous research methodologies to evaluate impact and identify potential harm to users as well as broader groups of people.","The key risk is the degradation of the TVS verification to degrade overtime based on the parameters of assessment for comparing images to templates. This was mitigated by testing the threshold for the biometric matching extensively with a variety of face types for several months to establish a match threshold for the identification. The algorithms have been trained on hundreds of millions of data points from a combination of public and private data sources. The underlying data covers a wide variety of demographics (e.g., age, race, gender, etc.) representative of the general population. During model training and refinement shortcomings (e.g., certain combinations of flesh tones and lighting, facial hair variations, etc.) were compensated for. The algorithms are industry leading as measured by NIST. This mitigates the risk of poor performance and algorithmic discrimination. ",Yes – by the CAIO,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,"Online - in the terms or instructions for the service.,In-person,Other",Facial matching and privacy notices are posted where facial recognition technology is in use.,No,The threshold for the biometric matching was tested extensively with a variety of face types for several months to establish a match threshold for the identification.,"Direct user testing,General solicitations of comments from the public",,Yes,,Yes, Silicon Valley Innovation Program (SVIP) Language Translator,Department of Homeland Security,DHS,USCG,Government Services (includes Benefits and Service Delivery),,None of the above.,"A handheld voice translator could be used to communicate the questioning of vessels in high-tension situations where a USCG vessel does not have a translator available, or they encounter different or multiple languages. In cases where a rescue is involved, being able to determine critical information such as the number of people aboard and status of life saving equipment is vital to commanders in their decision-making process. Once the USCG boarding team makes contact with the occupants of the boarded vessel, they must be able to communicate and give orders to the occupants of the vessel who may not speak English or in many cases speak multiple languages. The development of an offline language translator could enable accurate and swift translation of orders and directions which are critical to the safety and security of the boarding team and the occupants of the vessel.",Real time language translation.,Retired,,,,,N/A - Active as R&D only.,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Adaptive Risk Model for Inspected Small Passenger Vessels,Department of Homeland Security,DHS,USCG,Mission-Enabling,,None of the above.,"The Small Passenger Vessels (SPV) Safety Task Force leveraged machine learning and subject matter expert input to create an adaptable, quantitative analysis tool that reliably identifies the key drivers of marine casualties and calculates a risk score for each vessel within the largest segment of the U.S.-inspected fleet. A vessel risk score is computed with an analytic model which leverages logistic regression and basic machine learning. The effort directly improved marine inspector allocation, targeted risk and enhanced the governance necessary to increase passenger safety across the country.",Numerical score that compares vessels predicted safety risk relative to each other.,Operation and Maintenance,Neither,11/1/2019,11/1/2019,1/1/2021,,Developed in-house.,,No,,,No,,No,No,Yes,,"Commercial vessel profiles including: engineering, life saving, propulsion, fire protection, manning, operating routes, plan review, and USCG inspection activity details.","Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,No,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Intelligent Document Processing (IDP) for I-539 Form Digitization,Department of Homeland Security,DHS,USCIS,Mission-Enabling,,None of the above.,"IDP for the I-539 makes use of an AI-enhanced tool to identify, categorize, and create separate images for each document type submitted as part of the 539 benefit application. Prior to implementation of this use case, all pages of a 539 application were scanned and stored as a single document in the content management system. The benefit is reduced case processing time for adjudicators by identifying and classifying supporting documents for ease of use. An additional benefit is to bring digital images into compliance with NARA standards.","Input - one digital file comprised of all pages of a 539 benefit application. Output - multiple digital files comprised of the individual documents submitted as part of the 539 benefit application. These will include the 539 form, any other USCIS forms, and image files of other supporting documents such as Passports, Driver's license, Marriage Certificate, Bank Statement, etc. All pages of the original digital file are accounted for and stored. Any pages not identified by the tool are referred to a human for document type resolution.",Implementation and Assessment,Neither,9/30/2023,10/1/2023,,,Developed with contracting resources.,70SBUR23Q00000186,No,,,No,,Yes,Yes,Other,Model development is done with test data and input from adjudication Subject Matter Experts to identify form types,"CMS/STACKS test data is used to train the model. This data is comprised of digital images of blank USCIS forms and common supporting forms (Marriage License, Driver's License, Passport, ect.) generated using fake information such as Mickey Mouse and Donald Duck in place of PII.","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,No,No – agency does not have access to source code.,,Yes,Content Management Services (CMS),Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Large Language Models for an Officer Training Tool,Department of Homeland Security,DHS,USCIS,Mission-Enabling,,None of the above.,"The LLM for an Officer Training tool, will use GenAI to improve the way the agency trains immigration officer personnel. The tool will generate dynamic, personalized training materials that adapt to officers’ specific needs and ensure the best possible knowledge and training on a wide range of current policies and laws relevant to their jobs.","The AI outputs human-like conversation in a text format, similar to other AI chatbots, but specially trained and tuned for RAIO Officer training.",Implementation and Assessment,Neither,10/20/2023,11/1/2023,,,Developed with contracting resources.,70SBUR25F00000003,No,,,No,,No,Yes,No,,"The USCIS Officer Training Tool does use Proprietary/Private but Not Sensitive data, including training materials, internal guidance documents, and policies that are proprietary to the USCIS or the training program fall under this category. While not publicly accessible, this data is not considered sensitive but should still be treated with a degree of confidentiality to maintain the integrity of the training program.","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, User Entity and Behavior Analytics (UEBA) for Security Operations (SecOps) Anomaly Identification,Department of Homeland Security,DHS,USCIS,Mission-Enabling,,None of the above.,"UEBA's purpose is to review USCIS system logs to determine when an entity is performing actions that are anomalous. An entity can be classified as a workstation, server or an internal USCIS system account. The UEBA ingests logs from systems to perform analytics based off of models that are manually created and maintained. UEBA uses the models to apply a risk score to the entity which the risk score is then used to create a case (or ticket) for Security Operations analyst review. The AI reviews the action of the analyst to adjust the risk scoring for future events. Output would assist in prioritizing cyber events for further manual investigation.",Output of the Machine Learning is an alert with all artifacts for the SOC to investigate. The alert is used as a recommendation to prioritize specific investigations in the SOC ticket queue.,Acquisition and/or Development,Neither,8/18/2023,8/18/2023,,,Developed with contracting resources.,70SBUR24F00000168,No,,,,,,,No,,Data used to tune models is USCIS internal system logs.,"Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",None,,,,,Yes,ESS,6-12 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, I-765 - USCIS Face Capture Mobile App,Department of Homeland Security,DHS,USCIS,Law & Justice,,None of the above.,This will allow the user to complete the biometric verification requirement without having to attend an appointment at am Applicant Support Center. This reduces the burden on the beneficiary as well as reducing demands on USCIS Applicant Service Center resources.,"Face detection locates human faces in visual media such as digital images or video. When a face is detected it has an associated position, size, and orientation; and it can be searched for landmarks such as the eyes and nose which are returned as numerical values. ELIS Photo Validation service sends response back to user based on the pre-defined quality checks if the uploaded photo meets USCIS requirements. I-765 - USCIS Facial Recognition through IDENT (1:1 Face Recognition/Validation) returns a match or no match response from IDENT.",Acquisition and/or Development,"Rights-Impacting ",10/1/2023,10/1/2023,,,Developed with contracting resources.,70SBUR23F00000138,Yes,U.S. Citizenship and Immigration Services – DHS ,Enable increased self-service through an expansion of digital costumer service tools,,,,,Other,The applications is utilizing an ML model trained by commercial providers and thus data sets were not required.,"N/A - The Agency does not own any data used to train, fine-tune, or evaluate the use case.","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ","Race/Ethnicity,Sex/Gender,Age",,,,,No,,6-12 months,Yes,,No,,Other,N/A,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Case Processing Improvements in FDNS-DS NexGen,Department of Homeland Security,DHS,USCIS,Law & Justice,,None of the above.,"In the future, USCIS may consider using artificial intelligence (AI) / machine learning (ML) FDNS-DS NexGen or other systems may data from other applications for FDNS-DS NexGen or other systems to aid in investigative work, enhance investigative case prioritization, and detect duplicate case work. USCIS may consider and explore investigating whether it would be useful to integrate AI/ML into the predictive modeling for potential future system enhancements, also integrate AI/ML into the predictive modeling for future system enhancements, working side-by-side with the business stakeholders to develop best practices. Fraud occurs in numerous ways; being able to discover and detect persons with multiple identities allows for more comprehensive investigations, reduces investigative cycle time, and improves performance.","The requirements for this AI use case are currently unspecified, however, future outputs may include recommendations, prioritization/rankings, and anomaly detection pieces to assist with fraud detection and management.",Initiated,Neither,9/20/2022,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, USCIS Translation Service,Department of Homeland Security,DHS,USCIS,Government Services (includes Benefits and Service Delivery),,None of the above.,"Document Translation: Integrate AI and GenAI models trained on relevant subject matter (e.g., immigration law, visa/immigration applications, family/adoption certificates, and other sourced processing materials) to provide fast, accurate translation of written or other digital documents in various languages. Real-time Interpretation: Utilizing AI and GenAI-powered speech-to-speech, speech-to-text translation tools for efficient communication, consultations, and other interactions within DHS. Across all DHS components the need to support language translation and transcription is crucial for operations and adjudications.",Translated documents and translated languages.,Initiated,"Rights-Impacting ",1/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Verification Match Model,Department of Homeland Security,DHS,USCIS,Mission-Enabling,,None of the above.,"Leveraging AI in the USCIS verification matching process of known records across systems is beneficial because it streamlines existing USCIS review by 1) improving associated system accuracy, 2) reducing human-error by automating person-and-record match scoring, and 3) matching at a higher volume than traditional tools or manual processes can capably achieve.",A recommendation and score that indicates person-and-record match probability used by verification systems (E-verify and SAVE) to improve accuracy in initial system response.,Initiated,"Rights-Impacting ",7/12/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Sentiment Analysis -FOD Field Offices Complaints and Reviews,Department of Homeland Security,DHS,USCIS,Mission-Enabling,,None of the above.,"This system indicates the positive or negative feelings people express in their feedback to the U.S. Citizenship and Immigration Services (USCIS). Survey results categorized to specify the sentiments as positive negative or neutral tone in an excel dashboard.",A graph which categorizes the data into different sentiments using databricks dashboard to see how the customer service can be improved. Does not give any recommendation or decision.,Initiated,Neither,10/29/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Asylum Text Analytics,Department of Homeland Security,DHS,USCIS,,,None of the above.,"USCIS oversees lawful immigration to the United States. As set forth in Section 451(b) of the Homeland Security Act of 2002, Public Law 107-296, Congress charged USCIS with administering the asylum program. USCIS, through its Asylum Division within the Refugee, Asylum & International Operations Directorate (RAIO), administers the affirmative asylum program to provide protection to qualified individuals in the United States who have suffered past persecution or have a well-founded fear of future persecution in their country of origin, as outlined under Section 208 of the Immigration and Nationality Act (INA), 8 U.S.C. § 1158 and Title 8 of the Code of Federal Regulations (C.F.R.), Part 208. Generally, an individual not in removal proceedings may apply for asylum through the affirmative asylum process regardless of how the individual arrived in the United States or his or her current immigration status by filing Form I-589, Application for Asylum and for Withholding of Removal. The ATA capability employs machine learning and data graphing techniques to identify plagiarism-based fraud in applications for asylum status and for the withholding of removal by scanning the digitized narrative sections of the associated forms and looking for common language patterns. ","This use case is inactive. Please see the ""intended purpose and expected benefits"" for a full description of this use case. ",Retired,,1/25/2019,1/25/2019,10/1/2019,10/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Biometrics Enrollment Tool (BET) Fingerprint Quality Score,Department of Homeland Security,DHS,USCIS,,,None of the above.,"Duplicate with BET Maximization. The Biometrics Enrollment Tool (BET) team has incorporated the National Institute of Standards and Technology (NIST) Fingerprint Image Quality 2 (NFIQ2) algorithm (a trained machine learning algorithm) for scoring fingerprints (https://www.nist.gov/services-resources/software/nfiq-2) into the BET application. This algorithm takes a fingerprint image and assigns a score between 0 and 100, with 100 indicating the best quality fingerprint image that could be obtained. The higher the score, the more likely it is that the fingerprint will match when captured again.","This use case is inactive. Please see the ""intended purpose and expected benefits"" for a full description of this use case. ",Retired,,,,,N/A -Consolidated into another use case.,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Timeseries Analysis and Forecasting,Department of Homeland Security,DHS,USCIS,,,None of the above.,"USCIS is the component within DHS that oversees lawful immigration to the United States. That means USCIS receives, processes, and maintains all applications for admission for Lawful permanent residents (LPRs), or adjustments to LPR status. Also known as “green card” holders, LPRs are non-citizens who are lawfully authorized to live permanently within the United States and are required to fill out Form I-90, Application to Replace Permanent Resident Card (Green Card). Since there has been a considerable influx of green card applications, USCIS used a combination of exploratory data analysis to determine the most used categories for applicants submitting I-90's and machine learning to create predictions of workloads. As a follow-on, USCIS used Autoregressive Integrated Moving Average (ARIMA) models on the I-90 form, which allowed the prediction of the total number of forms for a 2-year period. ARIMA is one of the easiest and effective machine learning algorithms to perform time series forecasting. This capability has been deployed in production for more than a year. This model was eventually enhanced using ML model to have better reusability and performance.","This use case is inactive. Please see the ""intended purpose and expected benefits"" for a full description of this use case. ",Retired,,1/1/2020,1/1/2020,,6/1/2022,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Testing Performance of ML Model using H2O,Department of Homeland Security,DHS,USCIS,,,None of the above.,"USCIS is the component within DHS that oversees lawful immigration to the United States. That means USCIS receives, processes, and maintains all applications for admission for Lawful permanent residents (LPRs), or adjustments to LPR status. Also known as “green card” holders, LPRs are non-citizens who are lawfully authorized to live permanently within the United States and are required to fill out Form I-90, Application to Replace Permanent Resident Card (Green Card). Since there has been a considerable influx of green card applications, USCIS used a combination of exploratory data analysis to determine the most used categories for applicants submitting I-90's, and machine learning to create predictions of workloads. USCIS used the H20 machine learning model to allow USCIS analysts to build and run several machine learning models on big data in an enterprise environment and identify the model that performs the best. It has already been successful in identifying the most accurate model for the I-90 Form Timeseries Analysis and Forecasting use case. ","This use case is inactive. Please see the ""intended purpose and expected benefits"" for a full description of this use case. ",Retired,,,,,10/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, USCIS Consular Consolidated Database (CCD) Facial Recognition (FR) On Demand Report (VISA Only),Department of Homeland Security,DHS,USCIS,Mission-Enabling,,None of the above.,USCIS uses the Facial Recognition (FR) on Demand report (Visa only) to combat fraud by benefit applicants whose fingerprints are not in IDENT but who may have photos in the Department of State's (DoS) Consular Consolidated Database (CCD) that predate the fingerprinting of visa applicants. ,Match or no match response.,Operation and Maintenance,"Rights-Impacting ",,,4/1/2019,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,CCD is operated by the DoS Bureau of Consular Affairs. DHS works closely with DoS on risks and mitigations.,,,,,,,,,,,,,, Sentiment Analysis - Employee Satisfaction Surveys,Department of Homeland Security,DHS,USCIS,,,None of the above.,Survey results cosed to specify the sentiments as positive negative or neutral tone in a excel dashboard,A graph which categorizes the data into different sentiments using datanricks dashboard how the customer service can be improved. Does not give any reccomendation or decision.,Retired,,1/1/2019,,1/1/2019,10/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Sentiment Analysis - ELIS Case Notes,Department of Homeland Security,DHS,USCIS,,,None of the above.,Survey results cosed to specify the sentiments as positive negative or neutral tone in a excel dashboard,A graph which categorizes the data into different sentiments using datanricks dashboard how the customer service can be improved. Does not give any recommendation or decision.,Retired,"Rights-Impacting ",,,,10/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Predicted to Natzuralize,Department of Homeland Security,DHS,USCIS,,,None of the above.,The Predicted to Naturalize model predicts when Legal Permanent Residents would be eligible to naturalize and attempts to provide a current address. This model could potentially be used to send correspondence to USCIS customers of their resident status and notify others of potential USCIS benefits.,"This use case is inactive. Please see the ""intended purpose and expected benefits"" for a full description of this use case. ",Retired,,1/1/2023,1/1/2023,,N/A - This use case was never initiated and never formally retired.,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, I-485 Family Matching,Department of Homeland Security,DHS,USCIS,,,None of the above.,"I-485 Family Matching is designed to create models to match family members to underlying I-485 petitions. The underlying immigrant petition defines if the I-485 is employment-based or family-based. It also has information about the visa classification and priority date which, when compared against the Department of State’s monthly Visa Bulletin, helps predict visa usage. It is difficult to match an I-485 to its underlying immigrant petition, because the only available field on which to match is the Alien-number (A-number). This number is not always present on the immigrant petition, and name/date of birth matching is not as reliable.  The goal of I-485 Family Matching is to leverage AI to create connections more confidently between petitioners and their families based on limited data.  Additionally, it was intended to help identify and group I485s filed by family members, as well as gather up the many ancillary forms they may have been pending (such as I765, I131). Similar to immigrant petition matching, it can be difficult to match up I485s filed by family members. In these cases, the only similar fields are a common address. Efforts have been made in the past to identify family members by address, but it is effective only to a point. This use case explored the technical feasibility of using an AI model to help make working with this data more reliable (as well as group individual petitioners, their families, and other helpful associated data together for faster and more accurate processing).  Work on this use case did not go beyond ideation and no AI model was built (and no output was rendered) after the proof-of-concept established that the desired end-state was infeasible with the technology available at that time.","This use case is inactive. Please see the ""intended purpose and expected benefits"" for a full description of this use case. ",Retired,,1/1/2023,1/1/2023,,N/A - This use case was never initiated and never formally retired.,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Topic Modeling on Request For Evidence (RFE) Data Sets,Department of Homeland Security,DHS,USCIS,,,None of the above.,"Builds models that identify lists of topics and documents that are related to each topic. Topic modeling provides methods for automatically organizing, understanding, searching, and summarizing text data. It can help with the following: discovering hidden themes in the collection. classifying the documents into the discovered themes. ","This use case is inactive. Please see the ""intended purpose and expected benefits"" for a full description of this use case. ",Retired,,1/1/2021,1/1/2021,,10/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, I-539 approval prediction,Department of Homeland Security,DHS,USCIS,,,None of the above.,"Builds models that identify lists of topics and documents that are related to each topic. Topic modeling provides methods for automatically organizing, understanding, searching, and summarizing text data. It can help with the following: discovering hidden themes in the collection. classifying the documents into the discovered themes.","This use case is inactive. Please see the ""intended purpose and expected benefits"" for a full description of this use case. ",Retired,,1/1/2021,1/1/2021,,6/1/2021,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Text Analytics Data Science Sentence Similarity Model,Department of Homeland Security,DHS,USCIS,Law & Justice,,None of the above.,"Text Analytics augments the tedious and time-consuming manual process to identify potential fraud, national security, and/or public safety concerns and enables the identification of such concerns across jurisdictional boundaries. It increases the integrity of immigration programs, strengthens officers’ confidence in their work, and contributes to the reduction in customer wait times.","Text Analytics does not make predictions, recommendations, or decisions. It is merely a research tool that identifies potential patterns, while remaining agnostic as to whether those patterns identify potential fraud, national security, and/or public safety concerns. Instead, trained staff evaluate the patterns to determine whether they identify potential concerns and then validate and/or invalidate those potential concerns through the course of their investigations or adjudications.",Operation and Maintenance,"Rights-Impacting ",2/1/2019,2/1/2019,11/1/2019,,Developed with contracting resources.,70SBUR22F00000116,No,,,No,,Yes,Yes,Yes,,"Text Analytics stores information extracted from benefit forms and supporting documents, focusing on the narrative portions of those documents.","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Text Analytics,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,Yes,"Impact evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution and has utilized randomized experiments with a control group or other counterfactual, social systems analysis, or other rigorous research methodologies to evaluate impact and identify potential harm to users as well as broader groups of people.","There is small risk of false positives or false negatives due to the model. The risk is mitigated through a manual review of any information produced from the tool. Text Analytics is a decision support tool. Text Analytics does not make recommendations of fraud or benefit / adjudication decisions, any decisions made from information stored in the tool is conducted through a manual review by a USCIS employee.",Yes – by the CAIO,"Established Process of Machine Learning Operations: Alongside automated testing and drift detection, model re-training and re-deployments are supported by continuous integration pipelines that are managed by machine learning and data engineers on the platform, adapting work done by data science team into repeatable scripts for re-training and re-testing a model once deployed.",No - Some individual decisions or actions require direct human oversight.,,N/A - individuals are not interacting with the AI for this use case,,No,N/A,None of the above,,Yes,,"No – This AI use case is not subject to the opt-out requirement because the AI functionality is solely used for the prevention, detection, and investigation of fraud or cybersecurity incidents, or the conduct of a criminal investigation.", Biometrics Enrollment Tool (BET) Fingerprint Maximization,Department of Homeland Security,DHS,USCIS,Law & Justice,,None of the above.,"BET assists in determining if the fingerprint taken is good enough quality to pass the FBI fingerprint check process. It provides immediate feedback when a set of prints is likely to be rejected by the FBI by incorporating machine learning models into the BET application. The FBI will not disclose their quality grading criteria for fingerprints, leaving BET with the responsibility of determining quality to prevent unnecessary secondary encounters with applicants.","Numerical Fingerprint Quality score, which is compared against fingerprint quality thresholds (per finger and per set of fingerprints) to align with FBI specifications",Operation and Maintenance,Neither,6/1/2021,5/23/2023,1/1/2024,,Developed with contracting resources.,70SBUR23F00000126,No,,,No,,Yes,Yes,Yes,,"Internal data from BET data capture into Databricks lakehouse, numerical values representing fingerprint quality scores determined by the BET system outside of the AI workflow.","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Biometrics Enrollment Tool,Less than 6 months,No,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, ELIS Evidence Classifier Machine Learning (ML) Tagging Solution,Department of Homeland Security,DHS,USCIS,Government Services (includes Benefits and Service Delivery),,None of the above.,"To enable end users to navigate directly to the page(s) containing evidence documents of interest instead of sifting through large PDF documents. Evidence tagging intends to accelerate case processing by identifying specific types of documents (e.g., I-589, passport photo spread, marriage certificate) and applying a metadata tag to that document object in ELIS. This way, when a user opens a case with potentially hundreds of pages of evidence documents, rather than scrolling through them one at a time to find a specific document of interest, they have clickable ""bookmarks"" in the UI generated from these tags that will jump directly to the corresponding page.","Tagged evidence. The system inputs an image (scanned document from Lockbox) and outputs either a specific label, such as ""Border Crossing Card - Front,"" or no label if that document is not recognized as one of the classes.",Operation and Maintenance,Neither,11/20/2019,11/20/2019,9/1/2020,,Developed with contracting resources.,70SBUR23F00000120,No,,,No,,Yes,No,Yes,,Applicant submitted scanned evidence documents sent to ELIS and Global for processing.,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,ELIS,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, Automated Name and Date of Birth (DOB) Harvesting from Existing Records,Department of Homeland Security,DHS,USCIS,Government Services (includes Benefits and Service Delivery),,None of the above.,To reduce the amount of adjudicative time spent manually harvesting aliases and dates of birth (DOBs) from identity history summary (idHS) report attached to the ELIS case as part of the Manual Name Harvesting Task during case processing.,"Eliminates need for manual review by extracting unique names and DOBs from IdHS documents and when names/DOBs already in ELIS, ANH will not suggest any names.This is still a human in the loop process. The ELIS user performing MNH tasks prompted to make a decision if the suggested names and DOBs are related to case hence are able to accept/reject the suggestions.",Operation and Maintenance,Neither,7/10/2020,7/10/2020,6/27/2022,,Developed with contracting resources.,70SBUR23F00000120,No,,,No,,Yes,No,Yes,,"1. To train this deep learning model, we’ve gathered the text from about 8,000 IdHS records. 2. Typically, annotation of these entities would be done by hand. But in this case, we’re able to leverage the manual name harvesting that’s already been completed by ELIS users and therefore systematically tag our training data.","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,ELIS,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, Automated Realtime Global Organization Specialist (ARGOS) for Company Registration Submissions to E-Verify,Department of Homeland Security,DHS,USCIS,Mission-Enabling,,None of the above.,"ARGOS sentiment analysis produces a risk score and keyword extraction identifies the keword category of interest to the VAC MPAs (management and program analyst) for the aggregated open-source information to help quickly identify any pertinent information to aid the MPAs in their open-source investigation of company applications. This saves potentially thousands of MPA man hours in open-source investigation and creates a single source-of-truth for each MPAs investigation of a company application. This, in turn, allows for quicker application processing and, if risk of company fraud exists, much faster referral processing time quickening the next-step referral to FDNS for further investigations.",Responses back to a user dashboard accessible internally only by VAC Management and Program Analyst (MPA) personnel. Keywords relating to the MPA's work interest are extracted if present and risk scores are assigned to the open-source collected information. The data is presented to the MPA on the GUI (graphical user interface) dashboard.,Operation and Maintenance,"Rights-Impacting ",6/16/2021,6/16/2021,8/12/2023,,Developed with contracting resources.,70SBUR24F00000163,No,,,No,,No,No,Other,The machine learning models are using publicly safe open-source data. They do not need agency data for training. The process utilized for ARGOS is transfer learning.,The fine-tuned dataset is collected from open-source queries from the Bing API connected to the ARGOS system. This is publicly available data that doesn't contain any PII.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,ARGOS (Automated Real-Time Global Organization Specialist),Less than 6 months,Yes,,No,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,Yes,"Impact evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution and has utilized randomized experiments with a control group or other counterfactual, social systems analysis, or other rigorous research methodologies to evaluate impact and identify potential harm to users as well as broader groups of people.","Data Collection Bias: search engine prioritization of certain content may skew results. Lack of Domain-Specific Accuracy: model tested on company data across different industries resulted in inconsistent performance. Limited Generalization to Unseen Data: model’s performance on validation datasets lower than on training data, indicating potential overfitting. Misinterpretation of Sentiment: instances of sarcasm/irony not recognized. All risks identified in testing and evaluation phases. ",Yes – by the CAIO,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,N/A - individuals are not interacting with the AI for this use case,,No,"Bias monitoring, fine-tuning data balancing, and statistical precision deviation monitoring have been put in place.",None of the above,,Yes,,"No – This AI use case is not subject to the opt-out requirement because the AI functionality is solely used for the prevention, detection, and investigation of fraud or cybersecurity incidents, or the conduct of a criminal investigation.", ELIS Card Photo Validation via myUSCIS,Department of Homeland Security,DHS,USCIS,Government Services (includes Benefits and Service Delivery),,None of the above.,"USCIS uses a system called ELIS to manage immigration requests, and it includes a Photo Validation Service to check if ID photos meet requirements. This helps ensure photos are correct before making ID cards, saving time and avoiding delays. The photo validation service uses a combination of computer vision techniques and machine learning models to validate photographs and ensure they meet the requirements, so these photos can be used in card production.",Response back to user based on the pre-defined quality checks if the uploaded photo meets USCIS requirements. Users still have the option to ignore the warnings and upload the photo.,Operation and Maintenance,Neither,12/9/2020,12/9/2020,3/15/2022,,Developed with contracting resources.,70SBUR23F00000120,No,,,No,,Yes,Yes,Yes,,"Applicant submitted scanned or digital photographs sent to ELIS for processing, used for creating custom models or fine-tuning open source models and evaluation.","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,ELIS,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, I-765 - USCIS Facial Recognition through IDENT (1:1 Face Recognition/Validation),Department of Homeland Security,DHS,USCIS,Law & Justice,,None of the above.,This will allow the user to complete the biometric verification requirement without having to attend an appointment at am Applicant Support Center. This reduces the burden on the beneficiary as well as reducing demands on USCIS Applicant Service Center resources.,Match or no match response from IDENT.,Operation and Maintenance,"Rights-Impacting ",10/3/2023,10/3/2023,11/12/2024,,Developed with contracting resources.,70SBUR23F00000126,Yes,U.S. Citizenship and Immigration Services – DHS ,Enable increased self-service through an expansion of digital costumer service tools,No,,Yes,Yes,No,,"N/A - Agency does not own any data used to train, fine-tune, or evaluate the use case.","Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Customer Profile Management System,Less than 6 months,No,,No,,Other,N/A,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,Yes,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.",Potential mismatch of face images and/or bias based on demographic data held by USCIS.   USCIS is in the process of developing a reporting mechanism for false negative matches that will highlight any disparate impacts on various demographics. USCIS will produce an internal annual overview of the facial recognition results and any data points USCIS has about demographics. The use case has been found to not lead to or perpetuate unlawful discrimination or bias.,Yes – by the CAIO,"Automated and Regularly Scheduled Updates: Some aspects of the plan to monitor the AI system post-deployment are automated including re-training of models after detecting drift; however, data science teams are still significantly involved in the monitoring and re-deployment process.",No - Some individual decisions or actions require direct human oversight.,,N/A - individuals are not interacting with the AI for this use case,,No,The development of the face matching AI by the vendor began several years ago and it has been continuously updated since. . The algorithms are industry leading as measured by NIST’s Face Recognition Technology Evaluation (FRTE) n:1 benchmark. All testing met or exceeded expectations. [Meets § 5(c)(iv)(B)],"General solicitations of comments from the public,Other",,Yes,,No – There is law or governmentwide guidance that restricts opt-out for this context. , Person-Centric Identity Services Deduplication Model,Department of Homeland Security,DHS,USCIS,Law & Justice,,None of the above.,"Using Machine Learning allows us to improve entity resolution as compared to rule based system. PCIS offers the ability to see a person's immigration history organized in one place. Specific benefits do or will include: an organized summary view of the identity with the individual's latest photo from PCIS; full immigration history including receipts associated with the applicant, regardless of case management system; mailing, physical, and safe history of the individual organized in reverse chronological order, allowing users to easily find the most recent address; and all identifiers associated with the applicant, including A-Numbers, FINs, SSNs, SSNs, ELIS account numbers, passport numbers, etc.","Numerical likelihood score which is used to determine if the record belongs to the individual. Likelihood scores are subjected to a high threshold (.98, maximum 1) to assess whether the record belongs to the individual.",Operation and Maintenance,"Rights-Impacting ",1/1/2020,1/1/2020,2/1/2023,,Developed with contracting resources.,47QTCA22D00AB,No,,,No,,Yes,Yes,Yes,,"USCIS-only data derived from 7 form-processing source systems including C3, ELIS, CPMS, GLOBAL, CIS2, AR-11, CAMINO.","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Person Centric Identity Services,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,Yes,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.","There is small risk of false positives or negatives, which are identified and sent to Manual Resolution Queue. The queue is processed by authorized and trained personnel. Human review is still done for the actual benefit or request being sought. AI is used to identify the person seeking the benefit or request.  ",Yes – by the CAIO,"Automated and Regularly Scheduled Updates: Some aspects of the plan to monitor the AI system post-deployment are automated including re-training of models after detecting drift; however, data science teams are still significantly involved in the monitoring and re-deployment process.",No - Some individual decisions or actions require direct human oversight.,,N/A - individuals are not interacting with the AI for this use case,,No,N/A - The model is agnostic to demographics.,Direct user testing,,No – it is not operationally practical to offer this.,The model is only an entity resolution tool.,Other,"No, the model is not an automated decision-making tool, but functions as a tool that plays a critical role in the entity resolution and surfacing of a person and all their associated records." Person-Centric Identity Services A-Number Management Model,Department of Homeland Security,DHS,USCIS,Law & Justice,,None of the above.,"The aim of this use case is to leverage machine learning to test the accuracy of PCIS to identify and manage associations between individuals and their assigned A-numbers, which is a unique 7, 8, or 9 digit number assigned to a noncitizen by DHS. The A-Number plays a critical role in surfacing of a person and all their associated records from across PCIS.","Numerical likelihood score which is used to determine the validity of the A# presented. Likelihood scores are subjected to a high threshold (.98, maximum 1) to assess whether the A# presented belongs to the individual.",Operation and Maintenance,Neither,1/1/2022,1/1/2022,7/1/2022,,Developed with contracting resources.,47QTCA22D00AB,No,,,No,,Yes,Yes,Yes,,"USCIS-only data derived from 7 form-processing source systems including C3, ELIS, CPMS, GLOBAL, CIS2, AR-11, CAMINO.","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Person Centric Identity Services,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, Identity Match Option (IMO) Tool for Record Compilation,Department of Homeland Security,DHS,USCIS,Mission-Enabling,,None of the above.,"USCIS uses the IMO tool to aid in person-centric research and analytics. More specifically, IMO is used to derive a single identity across multiple systems for each applicant or beneficiary who interacts with USCIS.","A user friendly dashboard is created to display results and shows the data pattern but does not allow for any prediction or decision making. IMO is a COTS product offered by Informatica. The product has a variety of “transformations” that can be used together to build a workflow based solution. There are a number of different algorithms (Soundex, Jaro, Hamming distance, etc.) which are available for use in performing identity resolution.",Operation and Maintenance,Neither,1/3/2017,3/13/2017,1/3/2018,,Developed with contracting resources.,70SBUR21F00000194,No,,,No,,No,No,No,,"N/A - The Agency does not own any of the data used to train, fine-tune, and/or evaluate performance of the model(s) used in this use case.","Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,No,No – agency does not have access to source code.,,No,,Less than 6 months,No,,No,,Other,N/A,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Criminal Investigations (OBIM),Department of Homeland Security,DHS,USSS,Law & Justice,,None of the above.,"The intended purpose of this AI is so that USSS INV personnel may submit available photographs or video stills of these unknown persons as probe images (facial images or templates searched against the gallery of an FRS) to other government agencies for comparison against their image galleries. The agencies will query their image galleries of known persons and may provide lists of potential matches. USSS INV personnel may use the potential matches to produce investigative leads which will assist in the further identification of victims or suspects. Additionally, USSS INV personnel may request another government agency to conduct a one-to-one comparison of two photographs or video stills for investigative use.",The system will query image galleries of known persons and may provide lists of potential matches. USSS INV personnel may use the potential matches to produce investigative leads which will assist in the further identification of victims or suspects.,Operation and Maintenance,"Rights-Impacting ",10/30/2023,1/1/2016,1/1/2017,,Developed with both contracting and in-house resources.,RUIM-25-00019,No,,,No,,Yes,Yes,Yes,,OBIM is the data steward for the data in the IDENT system. The data owners are the 45 US and international organizations that collect and submit to IDENT - including DHS Components.,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Automated Biometric Identification System (IDENT),Less than 6 months,Yes,,Other,N/A - OBIM has not developed models for FR/FC and does not perform training on FR/FC AI models,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,Yes,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.","The product was developed by the NEC Corporation using AI and Deep Machine Learning to train the algorithm. However, the current NEC product that is used by OBIM in the production environment does not use AI to continue to train the NEC algorithm on production data. The fact that OBIM/NEC do not use AI on production data to continue to train the algorithm significantly limits the risks associated with the use of AI and ML on the face candidate list process.",Yes – by the CAIO,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,Online - in the terms or instructions for the service.,,No,"OBIM is currently working with the DHS CIO's office and DHS Science and Technology (S&T) to deeply evaluate the NEC P1 version algorithm that is currently employed by OBIM for the face candidate service. The NEC NeoFace P1 algorithm has also already been included in the National Institute of Standards and Technology (NIST) face recognition vendor testing and evaluation process. ",None of the above,,Yes,,"No – This AI use case is not subject to the opt-out requirement because the AI functionality is solely used for the prevention, detection, and investigation of fraud or cybersecurity incidents, or the conduct of a criminal investigation.", GitHub Copilot for Code Modernization,Department of Commerce,DOC,BEA - Bureau of Economic Analysis,Mission-Enabling,,,Accelerate and improve the quality of code modernization projects at BEA,"First-draft of code converted from legacy languages into modern, supported, target language. Code is then reviewed and edited by humans prior to use in an application.",Operation and Maintenance,Neither,10/1/2023,1/1/2024,12/1/2024,,Developed with contracting resources.,1331L524F13210282,No,,,No,"GitHub Copilot is used to create a first-draft of code converted from a legacy langauge into a supported, modern, target language. Code is then reviewed and tested by a human analyst before being incorporated into an application. Application and outputs are then verified during the SDLC for the product, validating the quality of the code and data outputs prior to approval for use to generate statistics.",No,Yes,Yes,,N/A--We are doing no model training.,"Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",Other,Does not use any demographic variables.,No,No – agency does not have access to source code.,,Yes,BEA-ITS,Less than 6 months,Yes,,Other,N/A--COTS tool used for code conversion.,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,Yes,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.",If the code produced by this tool is not sufficiently reviewed and tested then the data outputs calculated by the final product may not be accurate and the quality of inputs to BEA's PFEI statistics may be compromised. Key risk identified through completion of a Risk Analysis performed by the ITSO and CIO team.,Planned or in-progress,"Automated and Regularly Scheduled Updates: Some aspects of the plan to monitor the AI system post-deployment are automated including re-training of models after detecting drift; however, data science teams are still significantly involved in the monitoring and re-deployment process.","Other – Immediate human intervention is not practicable; however, an appropriate fail-safe has been implemented. ",All code generated by GitHub Copilot is reviewed and tested by the analyst prior to incorporation into an application.,N/A - individuals are not interacting with the AI for this use case,,No,"N/A; COTS tool used for code conversion, no individual's information is input into a model.","N/A; COTS tool used for code conversion, no individual's information is input into a model.",,"N/A; COTS tool used for code conversion, no individual's information is input into a model.",,"N/A; COTS tool used for code conversion, no individual's information is input into a model.", Azure OpenAI Chat Prototype,Department of Commerce,DOC,BEA - Bureau of Economic Analysis,Mission-Enabling,,,"Provide staff with the ability to chat with an LLM model, comparable to ChatGPT, that is contained within BEA's network boundary.",Text and data anlysis,Initiated,Neither,10/12023,,,,Developed with contracting resources.,1331L524F13210282,No,,,No,Outputs from the tool will be reviewed by a human for accuracy and reliability prior to incorporating text into a final document,No,No,Yes,,N/A--We are doing no model training.,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",Other,Does not use any demographic variables.,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,No,No testing: No testing of the model to simulate performance in an operational environment has been conducted thus far.,If the outputs from the chat tool are not accurate and are not reviewed sufficiently by the human writing the prompt then the quality of the end document will be compromised. ,Planned or in-progress,No monitoring protocols have been established: Necessary infrastructure needed to perform monitoring of AI systems in production is not available and/or a plan to monitor models has not yet been established.,No - Some individual decisions or actions require direct human oversight.,,N/A - individuals are not interacting with the AI for this use case,,No,N/A; still in initiation phase,None of the above,,No – it is not operationally practical to offer this.,Individuals do not interact with and are not impacted by this use case.,Other,Individuals do not interact with and are not impacted by this use case. Sentinel One Purple AI,Department of Commerce,DOC,BEA - Bureau of Economic Analysis,Mission-Enabling,,,Faster and more throughout incident response within the BEA cybersecurity program,Log analysis,Initiated,Neither,7/1/2024,,,,Developed with contracting resources.,,No,,,No,"Outputs from the tool will be reviewed by a human for accuracy and reliability; actions based on data provided by the AI will be manual, not automated.",No,No,Yes,,N/A--We are doing no model training.,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",Other,Does not use any demographic variables.,No,No – agency does not have access to source code.,,No,,More than 12 months,Yes,,Other,N/A--COTS tool used for code conversion.,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,No,No testing: No testing of the model to simulate performance in an operational environment has been conducted thus far.,If the outputs of the AI analysis of BEA logs is not accurate then the timeliness and accuracy of BEA's incident response program will suffer,Planned or in-progress,No monitoring protocols have been established: Necessary infrastructure needed to perform monitoring of AI systems in production is not available and/or a plan to monitor models has not yet been established.,No - Some individual decisions or actions require direct human oversight.,,N/A - individuals are not interacting with the AI for this use case,,No,N/A; still in initiation phase,None of the above,,No – it is not operationally practical to offer this.,Individuals do not interact with and are not impacted by this use case.,Other,Individuals do not interact with and are not impacted by this use case. Real Time Classification for the Economic Census,Department of Commerce,DOC,CENSUS - U.S. Census Bureau,Other,Classification,None of the above.,"Two ML models (BEACON and SINCT) are applied in this Economic Census use case to perform real-time classification of Primary Business Activity (PBA), or NAICS, and Products, or NAPCS. For instance, in the 2017 Economic Census, if respondents could not identify their NAICS or NAPCS codes, they were able to provide a short, open-ended description of their business and the products and services they offer, referred to as “write-ins”. There were nearly 400,000 NAICS write-ins and 1,000,000 NAPCS write-ins, out of 3 million and 18 million total reported NAICS and NAPCS codes, respectively. While the number of write-ins is a small share of the total number of reported NAICS and NAPCS codes, these write-ins were previously assigned to a valid code by analysts using manual processes. BEACON and SINCT were developed to improve classification accuracy and reduce processing time and costs by providing real-time feedback to respondents given the business description they provide. In the 2022 Economic Census, as a result of BEACON and SINCT, NAICS and NAPCS write-ins were reduced by 58% and 78% to 168,000 and 220,000 write-ins, respectively.","For PBA code reporting, the respondent has an initial set of NAICS codes to choose from. If the respondent selects ""Other"" with a write-in, an API call to BEACON returns predictive results from the model with a ranked list of predicted NAICS options from which to select. For Products, if the respondent selects to ""Add products"", an API call to SINCT provides predictive results of products to add. In both cases, the ML model recommends codes based on scoring attributes, and the respondent is able to choose a relevant NAICS or NAPCS code from those results.",Retired,Neither,,,1/1/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Automated Change Detection,Department of Commerce,DOC,CENSUS - U.S. Census Bureau,Mission-Enabling (internal agency support),,None of the above.,"As part of the Census Bureau's efforts to maintain and update the Geographic Frame ahead of the 2030 Decennial Census, the Geography Division is using AI to identify changes in the built environment throughout the nation twice a year. With the areas of change we are extracting our own geographic features (roads, building footprints) to support improving coverage across the nation.",We use Google Earth Engine to analyze change in construction using Sentinel-2 imagery and recommend where to use high resolution imagery. The object detection provides roads and building footprints to improve our feature coverage.,Implementation and Assessment,Neither,1/4/2021,5/24/2023,8/1/2023,,Developed with both contracting and in-house resources.,47QTCA19D00MP,Yes,United States Census Bureau,Obtaining Census Bureau statistics online,No,N/A,No,No,Yes,,"The high resolution imagery for training our building footprints and road predictions is from Maxar, USG Plus through National Geospatial-Intelligence Agency Global Enhanced GEOINT Delivery system. We are evaluating extracted features produced from the high resolution imagery and fine-tuning the output with staff to correct any inconsistencies.","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Google Cloud Platform and Enterprise Data Lake,6-12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, School staff information extraction from web page text,Department of Commerce,DOC,CENSUS - U.S. Census Bureau,Education & Workforce,,None of the above.,This AI use case examines publicly available information on the web related to staff rosters at public and private schools. The goal is to improve quality and coverage of Teacher and Principal frame.,The AI in this use case predicts whether a link on a webpage leads to a staff directory and extracts person names and teacher titles from webpage text.,Implementation and Assessment,Neither,1/15/2021,1/15/2021,5/10/2023,,Developed with both contracting and in-house resources.,Brite Group,No,,,No,N/A,Yes,No,Yes,,School employment lists and web scraped data.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Census Bureau Demographic Frame Person-Place Model,Department of Commerce,DOC,CENSUS - U.S. Census Bureau,Mission-Enabling (internal agency support),,None of the above.,"The Demographic Frame’s Person-Place Model (PPM) uses machine learning to assign individuals to their most accurate residential locations. This model addresses a critical need for accurate “person-place” associations, enabling the Census Bureau to support a range of essential use cases—from demographic analyses to population estimates. By leveraging machine learning, the PPM generates predicted probabilities for each person-address record, refining location accuracy while accommodating diverse data sources. This innovation strengthens the Census Bureau’s mission to deliver reliable, high-quality data, benefiting both governmental operations and the general public by enhancing data precision, policy-making, and service delivery.","The Person-Place Model (PPM) in the Demographic Frame outputs predicted probabilities indicating the likelihood that a person resides at a specific address. These predictions are generated by a machine learning model that analyzes historical and current data to produce a probability score for each person-address record. These scores do not constitute a final decision or assignment; instead, they provide a flexible basis for determining residential locations based on the desired level of confidence or specific application needs. This output can be used as input to downstream processes, providing robust, data-driven estimates to support various Census Bureau use cases and enhance the accuracy of demographic analyses.",Acquisition and/or Development,Neither,10/28/2021,10/28/2021,,,Developed with both contracting and in-house resources.,1331L523D13OS0003,No,,,,,,,Yes,,American Community Survey data,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.","Age Race/Ethnicity Sex/Gender",,,,,No,,Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Race and Ethnicity Autocoding,Department of Commerce,DOC,CENSUS - U.S. Census Bureau,Mission-Enabling (internal agency support),,None of the above.,This AI use case aims to develop an automated methodology for coding race and ethnicity write-in entries. The goal is to improve quality and efficiency in the coding of race and ethnicity write-in entries.,The AI produces a predicted set of standardized race and ethnicity concept codes that align with the write-in response.,Acquisition and/or Development,Neither,12/1/2021,2/15/2022,,,Developed in-house.,,No,,,,,,,Yes,,Response and clerical data.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",Race/Ethnicity,,,,,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Information extraction for web scraped data for Group Quarters frame enhancement,Department of Commerce,DOC,CENSUS - U.S. Census Bureau,Mission-Enabling (internal agency support),,None of the above.,This AI use case examines publicly available information on the web related to a variety of Group Quarters (GQ) types. The goal is to improve quality and coverage of GQ frame.,"Prediction label that corresponds to one of the following : extracted capacity of the GQ, extracted address component of the GQ, and extracted concepts, such as dates and building names, identified from pretrained NER models.",Implementation and Assessment,Neither,12/1/2021,5/16/2022,8/16/2024,,Developed with both contracting and in-house resources.,Brite Group,No,,,No,N/A,No,No,Yes,,Address lists and web scraped data.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Current Population Survey (CPS) Name Screening Tool,Department of Commerce,DOC,CENSUS - U.S. Census Bureau,Mission-Enabling (internal agency support),,None of the above.,This AI use case processes and classifies entries in survey data collect to identify if the name provided is a valid entry. The goal is to support and enhance processing and future data collection efforts to efficiently and accurately identify responses with invalid names.,Classifications for name entries.,Acquisition and/or Development,Neither,3/10/2023,3/29/2023,,,Developed with both contracting and in-house resources.,Brite Group,No,,,,,,,Yes,,Sample of survey response data from the Current Population Survey (CPS),"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,,,,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Linkage and Matching Program (LaMP),Department of Commerce,DOC,CENSUS - U.S. Census Bureau,Mission-Enabling (internal agency support),,None of the above.,"This use case involves probabilistic record linkage to link multiple person-level datasets. The predicted links are used for downstream processing to enable research regarding economic, demographic, and other topics and improve quality and efficiency of multiple Census operations.",The AI produces a prediction regarding whether two records match. Output from the model is pairs of records above a minimum threshold indicating a match along with the score describing the similarity between the two records in the pair.,Acquisition and/or Development,Neither,3/31/2023,8/1/2024,,,Developed with both contracting and in-house resources.,"Brite Group, Harmonia",No,,,,,,,Yes,,Survey response data from decennial census operations.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.","Age Sex/Gender",,,,,No,,Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Census Research Exploration and Analysis Tool (CREAT),Department of Commerce,DOC,CENSUS - U.S. Census Bureau,Education & Workforce,,None of the above.,"The Census Research Exploration and Analysis Tool (CREAT) is an experimental data tool from the Center for Economic Studies (CES) at the US Census Bureau that uses natural language processing to analyze, categorize, and sort the economic research contained in the CES working paper series. The goal of this project is to help CES researchers, managers, and other internal stakeholders explore connections among existing research, form new collaborations, and separate research into discrete topics. Working papers are sortable by author, tag, and keyword. Natural language processing (NLP) techniques are used to extract tags (using spaCy) and keywords (using KeyBERT) from the paper's text, as well as to determine working paper similarity (using Doc2Vec). This NLP usage highlights the value of these tools and techniques for making connections across economic research publications.","keywords, tags, and recommendations based on similarity.",Implementation and Assessment,Neither,11/1/2023,12/1/2023,3/1/2024,,Developed in-house.,,No,,,Yes,N/A,No,No,No,,CES working paper metadata,"Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Dr. NAICS LLM,Department of Commerce,DOC,CENSUS - U.S. Census Bureau,Mission-Enabling (internal agency support),,Searching for information using AI.,The core problem the Dr. NAICS LLM addresses is the ability to accurately and consistently place a NAICS code for a given business description. Because NAICS is such a large classification system with a lot of nuance it is sometimes difficult for businesses to identify the correct business description within it. The AI makes suggestions to improve that classification decision when reading incoming emails and voicemails from the general public.,The output is a set of recommendations for which NAICS codes best align with a text description of a business activity. ,Implementation and Assessment,Neither,11/27/2023,11/27/2023,9/30/2025,,Developed with both contracting and in-house resources.,1333LC23C00000030 and 1333LC23C00000026,No,,,No,N/A,No,No,No,,N/A,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",Other,Does not use any demographic variables.,No,,,No,,Less than 6 months,Other,,No,In the process of migrating the Dr. NAICS LLM beta app from the Reveal Server to inside the Census AWS server environment.,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Automating Multilingual Census Data Processing: An AI and Transformer-Based Pipeline for Efficient Language Detection and Translation for Short-Text,Department of Commerce,DOC,CENSUS - U.S. Census Bureau,Mission-Enabling (internal agency support),,None of the above.,"The purpose of this AI-driven pipeline is to tackle the challenge of processing non-English, short-text responses in large-scale surveys and censuses, especially brief entries for race and ethnicity. Traditional language detection and translation systems often struggle with these minimal text responses, impacting data accuracy and inclusivity. This AI solution is designed to automate the entire multilingual data processing workflow, from language detection to translation, named entity recognition, and validation, using AI and transformer-based models along with natural language processing techniques. By achieving high accuracy even with limited context, the system reduces the need for human translators, increases processing speed, and guarantees fair and accurate representation of diverse populations. This innovation not only supports the agency’s mission to collect inclusive, representative data but also benefits the public by contributing to more precise demographic insights, ultimately aiding in resource allocation and policy making.","The AI system outputs a series of automated decisions and validated translations for short-text responses, such as race and ethnicity write-ins, within large-scale survey data. Specifically, it provides accurate language detection, corrects potential spelling errors, generates contextually accurate translations, and validates these translations against standardized labels using semantic similarity analysis. The system then selects the most accurate translation for each input, guaranteeing precise categorization and extraction of demographic data. These outputs streamline survey data processing by reducing the need for human intervention, allowing for real-time or near real-time responses that meet high standards of accuracy and inclusivity",Acquisition and/or Development,Neither,6/3/2024,7/8/2024,9/30/2025,,Developed in-house.,,No,,,,,,,,,"The agency used agency-owned decennial paradata to train, fine-tune, and evaluate the AI model’s performance. This paradata, collected from previous census responses, provided important interaction data that allowed the model to improve its language detection and translation accuracy, especially for short-text responses typical in race and ethnicity write-ins.","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups.",Race/Ethnicity,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, FAQ for SMaRT,Department of Commerce,DOC,CENSUS - U.S. Census Bureau,Government Services (includes Benefits and Service Delivery),,Searching for information using AI.,"The purpose of the FAQ search tool is to find the correct Census Bureau-provided answers to common questions from the public about the 2025 Census Test. The 2025 Census Test is a proposed test as part of the Small-Scale Response Test (SmaRT) program in decennial. The purpose of this program is to perform experiments using different methods to improve self-response rates and data collection. The SmaRT program accomplishes this using iterative tests with relatively small sample sizes. Four installments of the SmaRT program were fielded before the 2020 Census. Three small-scale tests have occurred so far this decade with the fourth test planned for 2025. There is an online questionnaire for each of the tests, with a census.gov landing page to access the online questionnaire. In the first three tests this decade, there were approximately 15 FAQs on the landing page that the user could read. For the 2025 Census Test, we would like to offer over 50 FAQs to answer the public’s questions via a search tool. Current public-facing search tools at the Census Bureau use exact match only. Depending on what users type to search, the correct FAQ might not appear because users do not always use the same terminology as the Census Bureau. This FAQ search tool built with machine-learning models will use semantic similarity (i.e., synonyms or misspellings) to find the correct answers. The answers it provides back to the public are census-approved answers, and not generative AI answers. If the 2025 Census Test is approved, this search feature will be on the landing page for the survey. Implementing this search tool is considered a proof-of-concept for the use of such a search feature. If it is successful, the search tool could be used in the 2026 Census Test on its landing page, or the 2028 Dress Rehearsal. Ultimately, if it is successful, the public will benefit by finding answers to common questions more easily and more accurately.","Description of what the AI system outputs, whether it’s a prediction, recommendation, decision, etc. Utilizing transfer learning from Large Language Models (LLMs) we utilize them to encode our Census approved FAQs to then compare to a user query. This user query is then encoded as an embedding from the same LLM model and then we take a distance calculation using cosine similarity to rank our reference materials. We then showcase the top 3 results depending if a threshold is met of .25 cosine similarity or above. It is a semantic search tool that ranks reference material.",Acquisition and/or Development,Neither,7/8/2024,9/9/2024,7/11/2025,,Developed in-house.,,No,,,,,,,,,We did not fine tune the LLM because it was just used for transfer learning. We created a question-and-answer pairing dataset to estimate search accuracy. We also selected FAQs that had a high use rate in the 2020 Census to populate the 2025 FAQ list.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Statistical package syntax development and debugging,Department of Commerce,DOC,CENSUS - U.S. Census Bureau,Education & Workforce,,None of the above.,"I. Interaction Terms Generation: I utilized ChatGPT to generate SPSS syntax for creating interaction terms in a 97 X 4 matrix (race by item). This was integral for conducting moderated multiple regression analyses, aiming to establish differential prediction based on continuous supervisor scores and binary supervisor status (1 = supervisor, 0 = non-supervisor). II. Regression Analyses: The AI facilitated the development of large amounts of complex statistical analysis syntax for both binary logistic regression and multiple regression analyses, incorporating the interaction terms effectively. III. Racial Codes Creation: Used the AI to write syntax that would generate unique racial codes based on participants' self-identifications from a ""select all that apply"" question, allowing for a nuanced analysis of racial group memberships. IV. Consistency Comparison: Used AI to generate command syntax (for statistical software) to compare self-identified race and other protected class standings between two separate surveys, assessing consistency. The AI-generated syntax marked consistency (1) versus inconsistency (0) for each protected class.",The output was SPSS syntax and recommendations about configuring the syntax,Implementation and Assessment,Neither,9/16/2024,9/16/2024,9/16/2024,,Developed in-house.,,No,,,No,N/A,No,No,No,,none,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,No,,,No,,Less than 6 months,Other,,Other,Just access to the internet,Yes,no model training involved,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, DSD Python Code Translation,Department of Commerce,DOC,CENSUS - U.S. Census Bureau,Mission-Enabling (internal agency support),,None of the above.,The AI purpose for this effort is to assist the translation of SAS to Python code for the DSD data processing and supporting systems. The effort is in support of the Census mission to migration to enterprise-wide solutions and reducing dependency on costly proprietary software products.,The expected AI final system outputs are the translated Python code that replicates the SAS code. Comparison of the system outputs will be performed until the expected results are achieved.,Acquisition and/or Development,Neither,10/2/2024,10/28/2024,,,Developed with contracting resources.,1333LC,No,,,,,,,No,,N/A,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",Other,Pre-trained model for translating SAS code,,,,No,,Less than 6 months,No,,No,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Census API GPT,Department of Commerce,DOC,CENSUS - U.S. Census Bureau,Other,"Statistical Information about People, Places, and the Economy",Searching for information using AI.,"The Census API GPT provides guidance and assistance in using the Census API to retrieve a wide range of statistical information. It helps users form accurate API requests for demographic, economic, housing, and geographic data queries.","The Census API GPT offers clear explanations of the available datasets, parameters, and methods to efficiently retrieve the desired data, ensuring that users can access comprehensive information across these key areas. When necessary, it asks for clarification to ensure the accuracy and relevance of the request, making the process of using the Census API straightforward for users of all technical backgrounds. It emphasizes the importance of specificity in queries to reduce errors and improve the relevance of the data retrieved, catering to both beginners and experienced users alike. Additionally, the Census API GPT displays the data returned by the API, providing a direct view of the output alongside the guidance on constructing the API call.",Initiated,Neither,10/24/2024,,,,,,,,,,,,,,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Natural Language Search for data.census.gov,Department of Commerce,DOC,CENSUS - U.S. Census Bureau,Other,"Statistical Information about People, Places, and the Economy",Searching for information using AI.,The current search algorithm on data.census.gov does not support true natural language search and is keyword-based. Leveraging a natural language search interface powered by an LLM that uses a RAG architecture to fetch responses to user queries has the potential to vastly improve the search experience and ensure that users of all backgrounds are able to quickly find the information they are looking for.,"The AI system would consist of a natural language (LLM) input that would then use RAG (retrieval augmented generation) to identify an answer to the user's query (either in the form of an actual statistic, a relevant dataset, or relevant filters). Once an answer has been identified, it would be returned to the user via a natural language (LLM) interface.",Initiated,Neither,10/24/2024,,,,,,,,,,,,,,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, "ACES CAPEX (structures, equipment, other) Machine Learning",Department of Commerce,DOC,CENSUS - U.S. Census Bureau,Mission-Enabling (internal agency support),,Inputting large amounts of data from paper forms into a digital system using AI.,"The annual capital expenditures survey used to ask respondents to classify structures, equipment, or ""other"". Machine Learning was implemented to classify these items better. ACES is no longer an active survey, and the machine learning is no longer active.",Decision on if an item is structures or equipment.,Retired,Neither,,,1/1/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, FirstNet Authority OCIO MS CoPilot Project,Department of Commerce,DOC,FIRSTNET - First Responder Network Authority,Mission-Enabling,,Improving the quality of written communications using AI tools.,Microsoft 365 Copilot can improve content creation and boost user productivity experience. Microsoft 365 Copilot in the Microsoft Office apps will likely assist in creating content more quickly and improving existing content. This helps the users to convey their ideas with reduced effort and unlocks the value of existing organization information and knowledge. ,Text and/or graphics generated by user-input.,Initiated,Neither,,,,,,,,,,,,,,,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, FirstNet Authority Communications Topaz Labs Photo Editing,Department of Commerce,DOC,FIRSTNET - First Responder Network Authority,Mission-Enabling,,Creating visually appealing presentations using AI-driven design suggestions.,"Utilize AI photo and video editing tools to improve picture quality, smooth video, and scale images without degredation.",Digital photo and video files.,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Global Business Navigator Chatbot,Department of Commerce,DOC,ITA - International Trade Administration,Diplomacy & Trade,,Searching for information using AI.,ITA’s Global Business Navigator ChatbotBeta uses artificial intelligence (AI) to answer your questions on the exporting process and the resources available for exporters.,"Deliver concise, accurate responses to frequently asked export questions Guide users to relevant resources within Export Solutions content Refer more complex inquiries to trade specialists for personalized support Provide multilingual assistance, broadening accessibility to diverse user groups",Implementation and Assessment,Neither,2/2/2024,4/18/2024,N/A,N/A,Developed with both contracting and in-house resources.,"1331L524A13ES0005 (MS EA) 1331L523P13500087 (Data Svsc) 1331L524P13500090 (AI COE)",Yes,International Trade Administration - DOC,Service details are forthcoming,Yes,The GBN Chatbot is restricted from referencing any content from the internet or outside of the approx. 160 pages of authoritative content created by ITA tradecraft experts on trade.gov (and as published on trade.gov/export-solutions).,No,Yes,No,,"In formulating responses to user queries, GBN is restricted to referencing only the approx. 160 pages of web content in the trade.gov/export-solutions section of trade.gov. To fine tune the chatbot, ITA will explore using temporarily retained AI responses to customer questions, in order to evaluate the chatbot's performance. ","Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",Other,Does not use any demographic variables.,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Microsoft Azure ,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,No,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.","Key risks for using the AI chatbot on Trade.gov include data privacy and security, accuracy and reliability, and compliance with legal and regulatory standards. They were identified through project scoping, stakeholder consultations, and legal assessments. Mitigations involve strict cybersecurity protocols, continuous testing, and adherence to legal regulations.",Yes – by the CAIO,"Automated and Regularly Scheduled Updates: Some aspects of the plan to monitor the AI system post-deployment are automated including re-training of models after detecting drift; however, data science teams are still significantly involved in the monitoring and re-deployment process.",No - Some individual decisions or actions require direct human oversight.,,Online - in the terms or instructions for the service.,,No,"The agency has ongoing monitoring and regular audits are performed to identify and address any emerging biases, ensuring equitable service for all users.",Other,This feedback was incorporated during beta testing to refine the chatbot. Continuous monitoring ensures the chatbot meets the needs of all users.,Yes,,Other,"If they ask to speak to a person, they will receive the following answer: ""To speak to someone, first find your local office and the contact information (name, phone number, and email address) for one of our experienced trade professionals. Whether you're looking to make your first export sale or expand to additional markets, we offer the expertise you need to connect with lucrative opportunities and increase your bottom line. Understand Local Resources and Assistance."" Feel free to ask any follow-up questions!" Generative AI Tools Pilot - Global Markets,Department of Commerce,DOC,ITA - International Trade Administration,Diplomacy & Trade,,Improving the quality of written communications using AI tools.,"Accelerate and streamline report production. Enable analysts to focus on higher-level, strategic tasks rather than repetitive reporting. Reduce costs and improve efficiency, leading to better client service and faster decision-making.","Generate initial drafts of market research reports incorporating public data sources Provide data-driven insights to identify top markets and recommend strategic directions Integrate with existing workflows, supporting analysts in refining and finalizing comprehensive, actionable reports",Acquisition and/or Development,Neither,2/3/2024,N/A,N/A,N/A,Developed with contracting resources.,Planned or in Progress,No,,,,"All AI outputs will be considered first drafts only, requiring staff review for accuracy before release. This provides a critical check to catch and correct any inaccuracies, preventing their wider dissemination. The information obtained through the generative artificial intelligence pilot will be subject to existing Information Quality Act guidelines.",No,Yes,No,,TBD,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",Other,Does not use any demographic variables.,No,No – agency does not have access to source code.,,No,TBD,Less than 6 months,Other,"Planning to use a COTS product, no IT infrastructure is required",Other,"Planning to use a COTS product, no IT infrastructure is required",Other,Planned or in-progress,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,Planned or in-progress.,No testing: No testing of the model to simulate performance in an operational environment has been conducted thus far.,N/A - Users will be required to take all reasonable action to validate response from the generative AI tool,Planned or in-progress,No monitoring protocols have been established: Necessary infrastructure needed to perform monitoring of AI systems in production is not available and/or a plan to monitor models has not yet been established.,No - Some individual decisions or actions require direct human oversight.,,N/A - individuals are not interacting with the AI for this use case,The use of the generative AI tool is not public-facing,No,N/A - The generative AI tool will not be leveraged in a way that results in this type of disparity,Direct user testing,,No – it is not operationally practical to offer this.,N/A - The use of the generative AI tool will be user-led,Other,Users will opt-in to the pilot Generative AI Tools Pilot - Enterprise & Solutions Architecture,Department of Commerce,DOC,ITA - International Trade Administration,Mission-Enabling,,Improving the quality of written communications using AI tools.,"Reduce manual effort in creating and managing architecture documentation Accelerate solution design Enhance overall operational efficiency and service quality","Assist in drafting architectural documents and reference models based on standardized templates Summarize complex data sets and highlight key insights for faster decision-making Generate initial technical documents Aid in compliance checks by referencing and applying relevant frameworks and standards",Acquisition and/or Development,Neither,5/17/2024,N/A,N/A,N/A,Developed with contracting resources.,Planned or in Progress,No,,,No,N/A,No,Yes,No,,TBD,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",Other,Does not use any demographic variables.,No,No – agency does not have access to source code.,,No,TBD,Less than 6 months,Other,"Planning to use a COTS product, no IT infrastructure is required",Other,"Planning to use a COTS product, no IT infrastructure is required",Other,Planned or in-progress,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,Planned or in-progress.,No testing: No testing of the model to simulate performance in an operational environment has been conducted thus far.,N/A - Users will be required to take all reasonable action to validate response from the generative AI tool,Planned or in-progress,No monitoring protocols have been established: Necessary infrastructure needed to perform monitoring of AI systems in production is not available and/or a plan to monitor models has not yet been established.,No - Some individual decisions or actions require direct human oversight.,,N/A - individuals are not interacting with the AI for this use case,The use of the generative AI tool is not public-facing,No,N/A - The generative AI tool will not be leveraged in a way that results in this type of disparity,Direct user testing,,No – it is not operationally practical to offer this.,N/A - The use of the generative AI tool will be user-led,,Users will opt-in to the pilot WAWENETS,Department of Commerce,DOC,National Telecommunications and Information Administration,,The algorithm produces estimates of telecommunications speech quality and speech intelligibility. The input is a recording of speech from a telecommunications system in digital file format. The output is a single number that indicates speech quality (typically on a 1 to 5 scale) or speech intelligibility (typically on a 0 to 1 scale).,,,,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Streamline Spectrum Activities,Department of Commerce,DOC,National Telecommunications and Information Administration,Other,Spectrum Management,None of the above.,Streamline NTIA Office of Spectrum Management (OSM) processes and enhance Federal spectrum management. This includes using Azure and CoPilot capabilities to reduce the level of effort and improve the timeliness of OSM and Federal spectrum management processes.,"OSM CoPilot provides General office productivity outputs. Additional outputs include orchestration of current spectrum management functionality, speeding processes by eliminating clicks and keystrokes while under the supervision of spectrum stakeholders.",Acquisition and/or Development,Neither,12/10/2024,12/10/2024,,,,,No,,,No,,No,,Other,NTIA spectrum data exists but requires transformation to AI-ready / machine-readable format ,NTIA Manual of Regulations and Procedures for Federal Radio Frequency Management,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",Other,Does not use any demographic variables.,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Spectrum Visualizations,Department of Commerce,DOC,National Telecommunications and Information Administration,Science & Space,Creating visual representations of data sets for reports and presentations using AI.,,,,Initiated,Neither,,,,,,,,,,,,,,,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Use of Lexis. Other searches for legal research ,Department of Commerce,DOC,National Telecommunications and Information Administration,Law & Justice,,,,,Initiated,Neither,,,,,,,,,,,,,,,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Grants Program Administration,Department of Commerce,DOC,National Telecommunications and Information Administration,Government Services (includes Benefits and Service Delivery),,,"Our team is exploring using AI to support our grant program administration, specifically supporting some AI-enable knowledge management for our internal team and supporting reviews of post award technical/progress/performance/financial reports to identify trends, identify inconsistencies, etc.  The latter may be more machine learning/compliance-based analytics and be on the fringe of AI.  ",,Initiated,Neither,,,,,,,,,,,,,,,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Grammarly,Department of Commerce,DOC,NIST - National Institute of Science and Technology,Education & Workforce,,Improving the quality of written communications using AI tools.,"Improving the quality of written communications, idea generation, refine tone and style. ","Grammar, punctuation, style improvements, content generation",Operation and Maintenance,Neither,2/3/2023,6/16/2023,1/17/2024,,Developed with contracting resources.,NNG15SC82B,No,,,No,,No,Yes,No,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",Other,Does not use any demographic variables.,No,No – agency does not have access to source code.,,Yes,GrammarlyGo,,No,,Other,There is a process in place to request a Grammarly account. GrammarlyGo does not use our data for training.,Other,Grammarly is a cloud app that we support and provision accounts for those that request a license,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,Yes,No testing: No testing of the model to simulate performance in an operational environment has been conducted thus far.,"Risks to NIST were determined through Assessment and Authorization process by NIST Security Control Assessors. They documented that the most significant risk to NIST is moderate-level data going to a cloud vendor that does not have a FedRAMP authorized package and is not currently in process to become FedRAMP. Grammarly does not meet Department of Commerce Security and Privacy Assessment and Authorization Handbook requirements. There are deficiencies restricting staff to US only personnel, specific documentation about use of FIPS 140-2 validated encryption, TIC 2.0 architecture implementation, and Grammarly is unable to send logs to DOC SIEM tools. This is mitigated in multiple ways: • Grammarly has multiple independent assessments, including a SOC 2 Type II assessment. • Grammarly provided other internal security documentation for review. • NIST data is only used by Grammarly to perform grammar recommendations. All data is purged from Grammarly servers within 3 hours. • NIST data is not used for training purposes. • Grammarly uses Amazon Web services for its infrastructure, which has FedRAMP Moderate authorization. • Grammarly uses Microsoft Azure AI for AI recommendations, which has FedRAMP High authorization. Other key risks are as follows: • Logins to Grammarly are logged and viewable by administrators, but the actual usage of Grammarly by the user is not logged. This is mitigated by Grammarly’s limited built-in data loss prevention (DLP) that is enabled. This prevents the following data types from going up to Grammarly for analysis: email addresses, URLs, phone numbers, credit card numbers, and Social Security numbers. ",Yes – by another appropriate agency office that was not directly involved in the system’s development,"Automated and Regularly Scheduled Updates: Some aspects of the plan to monitor the AI system post-deployment are automated including re-training of models after detecting drift; however, data science teams are still significantly involved in the monitoring and re-deployment process.",Yes - All individual decisions or actions are automated and may be rendered without direct human interaction.,,Other ,"Training was provided to end users on how to use the Grammarly application. If users have any questions, they submit a ticket through ServiceNow.",No,The model provides grammar and phrasing recommendations. All staff are presumed to be English subject matter experts and do not need to accept any proposed response from Grammarly. Staff are advised that they are ultimately responsible for the content of their responses.,None of the above,,Yes,,Yes, Science Data Portal Autosuggest Search ,Department of Commerce,DOC,NIST - National Institute of Science and Technology,Science & Space,,Searching for information using AI.,"Assist end users is discovery, access, and re-use of scientific, engineering and technical information provided by bureau","Dynamically generated terms, filtered and ranked for frequency",Operation and Maintenance,Neither,10/1/2021,10/1/2021,2/1/2022,,Developed in-house.,,No,,Enable increased self-service through an expansion of digital costumer service tools,Yes,N/A ; See https://www.nist.gov/director/nist-information-quality-standards,No,Yes,Yes,,NIST public data repository metadata corpus,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",Other,Does not use any demographic variables.,Yes,Yes – source code is publicly available.,https://github.com/usnistgov/oar-autocomplete,No,https://data.nist.gov,More than 12 months,No,,Yes,,Yes,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,No,"Impact evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution and has utilized randomized experiments with a control group or other counterfactual, social systems analysis, or other rigorous research methodologies to evaluate impact and identify potential harm to users as well as broader groups of people.",Risk is low to no risk in use,Yes – by another appropriate agency office that was not directly involved in the system’s development,No monitoring protocols have been established: Necessary infrastructure needed to perform monitoring of AI systems in production is not available and/or a plan to monitor models has not yet been established.,No - Some individual decisions or actions require direct human oversight.,,Online - in the terms or instructions for the service.,,No,N/A,Direct user testing,,Yes,System provides customer support via email submission for feedback and questions/issues related to system Search functionality ,Yes, FathomNet,Department of Commerce,DOC,NOAA - National Oceanic and Atmospheric Administration,Science & Space,Ocean imagery,None of the above.,Ocean image annotation and model building,,Acquisition and/or Development,Neither,Unknown,,Ongoing,,Developed with contracting resources.,,,,,,,No,No,,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Coastal Change Analysis Program (C-CAP),Department of Commerce,DOC,NOAA - National Oceanic and Atmospheric Administration,Science & Space, Landcover developement,None of the above.,Develop high resolution land cover data for the United States.,,Implementation and Assessment,Neither,7/7/1905,2015-2022,Ongoing,,Developed with both contracting and in-house resources.,,,,,,,No,No,Yes,,Existing NOAA C-CAP and other U.S. government authoritative geospatial datasets.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",Other,Does not use any demographic variables.,Yes,Yes – source code is publicly available.,https://github.com/NOAA-OCM/CCAP,,,,,,Yes,,,,,,,,,,,,,,,,,,,,,,, "CoralNet: Ongoing operational use, improvement, and development, of machine vision point classification",Department of Commerce,DOC,NOAA - National Oceanic and Atmospheric Administration,Science & Space,Coral reef conservation,None of the above.,Computer vision for coral reef monitoring and conservation efforts.,,Acquisition and/or Development,Neither,7/7/1905,,,,Developed with contracting resources.,,,,,,,No,No,,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Coral Reef Watch,Department of Commerce,DOC,NOAA - National Oceanic and Atmospheric Administration,,In production: more than 1 year,,,Remote sensing data interpretation and prediction,Initiated,Neither,,,,,,,,,,,,,,,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Steller sea lion brand sighting ,Department of Commerce,DOC,NOAA - National Oceanic and Atmospheric Administration,,In production: more than 1 year,,,"U-net model, using R and a package called TLC specifically designed for this project",Initiated,Neither,,,,,,,,,,,,,,,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, "CoralNet: Ongoing operational use, improvement, and development, of machine vision point classification.",Department of Commerce,DOC,NOAA - National Oceanic and Atmospheric Administration,,In production: more than 1 year,,,FCNN,Initiated,Neither,,,,,,,,,,,,,,,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, "BANTER, a machine learning acoustic event classifier",Department of Commerce,DOC,NOAA - National Oceanic and Atmospheric Administration,,In production: less than 1 year,,,,Initiated,Neither,,,,,,,,,,,,,,,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Operational satellite product quality monitoring AI-ready training data,Department of Commerce,DOC,NOAA - National Oceanic and Atmospheric Administration,Other,Satellite Data and Product Utility,None of the above.,"NOAA NDE and NESDIS Common Cloud Framework (NCCF) are current and future NOAA satellite product generation environments. There are hundreds of products based on a variety of national/international satellites (US, Europe and Japan) that are generated in near real time. The anomaly detections of the satellite product quality are still manpower intensive and in a traditional way. Nowadays, with the AI technology getting mature, the AI use case can be extended to this area of anomaly detection. In the traditional way, the maintenance programmers eyeball the product maps with 24/7 support which is stressful. On the other hand, AI has the strong capability of pattern recognition with high accuracy. The AI-based technique will relieve manpower and automate the alert system.","Detection/Identification of the satellite product anomalies, e.g. the bounding boxes locate the pre-defined error types. ",Acquisition and/or Development,Neither,11/6/2022,3/29/2024,N/A,N/A,Developed with contracting resources.,1332KP21DNEEB0011,No,,,No,N/A,No,No,No,,A variety of operational satellite products,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",Other,Does not use any demographic variables.,Yes,"Yes – agency has access to source code, but it is not public.",,No,,More than 12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,Planned or in-progress.,No testing: No testing of the model to simulate performance in an operational environment has been conducted thus far.,Lack of funding for Phase II development and transition from research to operations. ,Yes – by an agency AI oversight board not directly involved in the system’s development,No monitoring protocols have been established: Necessary infrastructure needed to perform monitoring of AI systems in production is not available and/or a plan to monitor models has not yet been established.,No - Some individual decisions or actions require direct human oversight.,,Email,,No,"Test, assess and feedback collection",Direct user testing,,Yes,,Other,"No – This AI use case is not subject to the opt-out requirement because the AI functionality is solely used for the prevention, detection, and investigation of satellite data product quality." LightningCast: AI for lightning prediction,Department of Commerce,DOC,NOAA - National Oceanic and Atmospheric Administration,Other,Hazard Prediction,None of the above.,"LightningCast provides prognostic information on lightning, before the threat is realized, thereby allowing for actions that protect life and property",Probability that lightning will be observed at a given location in the next 60 minutes,Implementation and Assessment,Neither,9/1/2019,9/1/2020,Scheduled for FY2025,N/A,Developed in-house.,N/A,No,,,No,N/A,No,No,No,,Data from the Geostationary Operational Environmental Satellites,"Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",,,Yes,Yes – source code is publicly available.,https://cimss.ssec.wisc.edu/csppgeo/lightningcast_v1.0b1.html ,No,,More than 12 months,Yes,,Yes,,Yes,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,Planned or in-progress.,"Impact evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution and has utilized randomized experiments with a control group or other counterfactual, social systems analysis, or other rigorous research methodologies to evaluate impact and identify potential harm to users as well as broader groups of people.",None,Yes – by another appropriate agency office that was not directly involved in the system’s development,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,Online - in the terms or instructions for the service.,,No,N/A,Direct user testing,,No – it is not operationally practical to offer this.,The AI does not provide a service. It provides guidance.,Other,This AI is a decision-support tool used by meteorologists. Utilization is completely optional. CyberSecurity Hardening & Automation,Department of Commerce,DOC,NOAA - National Oceanic and Atmospheric Administration,Other,CyberSecurity Information & Configuration,None of the above.,"Promptly analyze cybersecurity architecture, documentation, inventories, and vulnerabilities for remediation, automation opportunities, continuous monitoring, responding to data calls, and architectural risk assessments (e.g. Security Impact Analyses [SIAs]).","Risk Assessment, documentation generation, responses to data calls",Initiated,Neither,1/1/2024,6/1/2024,N/A,N/A,Developed in-house.,N/A,No,,,No,,No,,,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, AI based Community Radiative Transfer Model,Department of Commerce,DOC,NOAA - National Oceanic and Atmospheric Administration,Other,"Satellite radiance assimilation for operational eather forecasts, physical retrievals of environmental data records",None of the above.,Fast radiative transfer calculations for increasing satellite data needed by weathers forecasts and the generation of satellite products.,simulated satellite radiances and sensitivities (Jacobian) of satellite data,Acquisition and/or Development,Neither,7/1/2024,7/1/2024,1/1/2027,N/A,Developed in-house.,N/A,Yes,,,No,N/A,No,No,No,,Meteorological data such as profiles of temperature and water vapor,"Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",,,No,No – agency does not have access to source code.,,No,,More than 12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,Planned or in-progress.,No testing: No testing of the model to simulate performance in an operational environment has been conducted thus far.,Lack of funding,Yes – by another appropriate agency office that was not directly involved in the system’s development,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,N/A - individuals are not interacting with the AI for this use case,,No,"Test, assess and feedback collection",Direct user testing,,No – it is not operationally practical to offer this.,Very low risk to use this AI based application algorithm.,Yes, Operational Enterprise Cloud Mask,Department of Commerce,DOC,NOAA - National Oceanic and Atmospheric Administration,Science & Space,Operational satellite data product,None of the above.,Used to create internal ECM algorithm classifiers to optimize cloud detection over different surface types and atmospheric conditions,Cloud detection ,Implementation and Assessment,Neither,Pre-2021,Pre-2021,Pre-2021,N/A,Developed in-house.,N/A,No,,,,,No,No,No,,Data from operational satellites,"Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",,,Yes,Yes – source code is publicly available.,https://gitlab.ssec.wisc.edu/clavrx/clavrx-dev,No,,More than 12 months,Yes,,Yes,,Yes,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,Planned or in-progress.,"Impact evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution and has utilized randomized experiments with a control group or other counterfactual, social systems analysis, or other rigorous research methodologies to evaluate impact and identify potential harm to users as well as broader groups of people.",None,Planned or in-progress,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,Online - in the terms or instructions for the service.,,No,,Direct user testing,,No – it is not operationally practical to offer this.,The AI provides scientific guidance for cloud detection.,Other,This AI use is a component of an operational satellite data production algorithm. The product is used by atmospheric scientists. Utilization is completely optional. AI Snowfall Detection and Snowfall Rate estimation,Department of Commerce,DOC,NOAA - National Oceanic and Atmospheric Administration,Science & Space,,None of the above.,,,Initiated,Neither,,,,,,,,,,,,,,,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, AI Rain Rate estimation,Department of Commerce,DOC,NOAA - National Oceanic and Atmospheric Administration,Science & Space,,None of the above.,,,Initiated,Neither,,,,,,,,,,,,,,,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, AI Total Precipitable Water estimation,Department of Commerce,DOC,NOAA - National Oceanic and Atmospheric Administration,Science & Space,,None of the above.,,,Initiated,Neither,,,,,,,,,,,,,,,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, AI Cloud Liquid Water estimation,Department of Commerce,DOC,NOAA - National Oceanic and Atmospheric Administration,Science & Space,,None of the above.,,,Initiated,Neither,,,,,,,,,,,,,,,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, AI retrieval for patent search,Department of Commerce,DOC,USPTO - United States Patent and Trade Office,Mission-Enabling,,None of the above.,"Augmentation for next generation patent search tool to assist examiners identify relevant documents and additional areas to search. System takes input from published or unpublished applications and provides recommendations on further prior art areas to search, giving the user the ability to sort by similarity to concepts of their choosing.","System takes input from published or unpublished applications and provides recommendations on further prior art areas to search, giving the user the ability to sort by similarity to concepts of their choosing.",Operation and Maintenance,Neither,11/7/2018,10/31/2019,10/15/2021,,Developed with both contracting and in-house resources.,1333BJ24C00280005,No,,,No,,Yes,No,No,,"The models are trained on Patent Publication information, including US Patent Publications, that are publicly available.","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",Other,Does not use any demographic variables.,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,PPL-Patent Search Artificial Intelligence (PTOC-00-060-00),Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, AI use for CPC classification,Department of Commerce,DOC,USPTO - United States Patent and Trade Office,Mission-Enabling,,Searching for information using AI.,"Automatically assign patent utility classification (CPC) symbols to patent utility documents to reduce classification time, provide agency cost savings, and improve consistency of classification practices.",Predictions of CPC symbols relevant for each utility patent document inputted,Acquisition and/or Development,Neither,10/25/2017,10/9/2019,,,Developed with contracting resources.,1333BJ21D00280001 (ID/IQ contract number),Yes,United States Patent and Trademark Office,Applying for a patent,No,,No,Yes,Other,"While we have access, the use case data used is not contained in those databases. The use case uses USPTO owned classification (CPC) data on patent utility documents since 1976; USPTO owned patent document text data; agency published public data of CPC scheme and definitions.",The use case uses USPTO owned classification (CPC) data on patent utility documents since 1976; USPTO owned patent document text data; agency published public data of CPC scheme and definitions.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",Other,Does not use any demographic variables.,Yes,"Yes – agency has access to source code, but it is not public.",N/A,Yes,OD/BD master system,More than 12 months,Yes,N/A,Yes,N/A,Yes,N/A,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,No,"Benchmark evaluation: Testing of the AI model has involved the use of benchmarks (either publicly available or internally created) to estimate performance in real-world settings, but has not been tested in an operational environment. ",unable to meet performance quality metrics- will be identified by benchmark evaluation and future operation-environment testing,"Does not apply, use case is neither safety or rights impacting.","use case is not safety impacting or rights impacting; however, a monitoring process for AI performance is under development","use case does not impact safety or rights; however – Other, Immediate human intervention is not practicable; however, an appropriate fail-safe has been implemented. ","AI decisions provide initial CPC symbols to patent documents, the AI decisions are reviewed by appropriate patent technical experts in the patent examination process",Other ,Use case is not safety or rights impacting; however Agency has notified the public that AI is being explored for classification; appropriate notice to public will be provided before full implementation.,No,Planned performance monitoring to evaluate performance across technologies and patent applications,"direct user testing outreach to relevant labor organizations other",Autoclass has regularly reported to and received feedback from public stakeholders thru Patent Public Advisory Committee (PPAC) and is working with internal stakeholders for direct quality assessment/testing to benchmark AI performance,No – it is not operationally practical to offer this.,Use case is not safety impacting or rights impacting; however - Internal stakeholders do and will have the option to change the decision if they disagree; decision is not appeallable or contestable by external stakeholders as the decision.,Other,No - the use case has be determined to not impact safety or rights; there is no planned opt-out mechanism by individual as the AI use case is for agency internal functioning AI retrieval for TM design coding and image search,Department of Commerce,DOC,USPTO - United States Patent and Trade Office,Mission-Enabling,,Searching for information using AI.,"Clarivate COTS solution to assist examiner identification of similar trademark images, to suggest the correct assignment of mark image design codes, and to determine the potential acceptability of the identifications of goods and services.",System is anticipated to use both incoming trademark images and registered trademark images and output design codes and/or other related images.,Initiated,Neither,9/30/2021,,,,,,No,,,No,,,,,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Enriched Citation,Department of Commerce,DOC,USPTO - United States Patent and Trade Office,Government Services (includes Benefits and Service Delivery),,Summarizing the key points of a lengthy report using AI.,"Data dissemination system that identifies which references, or prior art, were cited in specific patent application office actions, including: bibliographic information of the reference, the claims that the prior art was cited against, and the relevant sections that the examiner relied upon. Use of AI to extract summarized information from unstructured text is providing operational cost savings. ",System extracts information from unstructured office actions and provides the information through a structured public facing API.,Operation and Maintenance,Neither,9/18/2017,11/7/2017,3/21/2019,,,,No,,,No,,,,,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,,,,,,,,,,,,,,,No – Agency did not request an extension for this use case.,No,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.",N/A - Not Safety or Rights impacting,,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",Yes - All individual decisions or actions are automated and may be rendered without direct human interaction.,,N/A - individuals are not interacting with the AI for this use case,,No,N/A - Not Safety or Rights impacting,Direct user testing,,Yes,,Yes, Inventor Search Assistant (iSAT),Department of Commerce,DOC,USPTO - United States Patent and Trade Office,Government Services (includes Benefits and Service Delivery),,Searching for information using AI.,"Service to help inventors ""get started"" identifying relevant documents, figures, and classification codes used to conduct a novelty search. Expected benefit was to provide continuity in service access to public search capabilities during COVID-19 related closure of in-person search assistance services. ","System takes a user entered short description of invention and provides a user selectable set of recommended documents, figures, and classification areas.",Retired,Neither,10/3/2019,2/3/2020,6/15/2021,10/18/2024,,,No,,,Yes,,,,,,,,,,,,,,,,,,,,,,,,No – Agency did not request an extension for this use case.,No,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.",N/A - Not Safety or Rights impacting,,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,None of the above,,No,N/A - Not Safety or Rights impacting,Post-transaction customer feedback collections,,Yes,,Yes, Patent Design Image Search (DesignVision),Department of Commerce,DOC,USPTO - United States Patent and Trade Office,Mission-Enabling,,Searching for information using AI.,A SaaS AI driven image search tool to support the examination of design applications. ,"The tool compares uploaded images from design applications and yields image results based on similarity thereby providing access, search, and retrieval of information from Industrial Design collections.",Implementation and Assessment,Neither,2022-11-16 (first formal action related to this project),2024-02-15 (contract kick-off); 2024-08-23 (signed Acquisition Plan),2024-10-16 (pilot licenses assigned),,Developed with contracting resources.,1333BJ24C00150001,No,,,No,,Yes,No,No,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,No – agency does not have access to source code.,,Yes,PTO-PPL-DV-DesignVision (PPL-PEC-02-00),Less than 6 months,No,,No,,No,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, USPTO Virtual Assistant (Public-facing chatbot),Department of Commerce,DOC,USPTO - United States Patent and Trade Office,Government Services (includes Benefits and Service Delivery),,None of the above.,Answers questions about Trademarks and Patents,Answers to Trademark and patent questions that the public has.,Operation and Maintenance,Neither,4/1/2021,5/1/2021,12/8/2022,,Developed in-house.,,Yes,United States Patent and Trademark Office,Applying to register a trademark; applying for a patent,Yes,The system has been screened by the Cyber security for compliance. It has acquired the Authority to operate (ATO) in 2023.,No,Yes,Other,All data dissiminated in the system is approved by Trademark or Patent law offices.,The model for Chatbot Virtual Assistant is trained on data pertaining to Trademark or Patent information that is publicly available on Trademark and Patent web pages.,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",Other,Does not use any demographic variables.,Yes,"Yes – agency has access to source code, but it is not public.","RASA 3.1, BERT",Yes,Chatbot-C,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,Yes,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.",Minimal risks were identified for Chatbot Virtual Assistant as the data that is dissiminated to the users are from preapproved dataset. Chatbot cannot answer questions outside of what it is trained on.,Yes – by the CAIO,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,Online - in the terms or instructions for the service.,,No,The greeting message clearly states not to use PII or demographic information ,General solicitations of comments from the public,,No – it is not operationally practical to offer this.,This does not apply to the Chatbot Virtual Assitant use case as all the outcomes of this system is predefined and approved.,Yes,The users can opt to speak to human operated help desk. Patent Automatic Document Code Determination,Department of Commerce,DOC,USPTO - United States Patent and Trade Office,Mission-Enabling,,Inputting large amounts of data from paper forms into a digital system using AI.,Quality control of customer applied document labels,Document Labels,Implementation and Assessment,Neither,2/1/2023,2/1/2023,To Be Determined (calendar 2025),,Developed with both contracting and in-house resources.,280-EIPL-0762 ,No,,,No,,No,No,Yes,,Publicly available open data of previous documents and their labels,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",Other,Does not use any demographic variables.,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,PE2E-DAV,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,Yes,"Impact evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution and has utilized randomized experiments with a control group or other counterfactual, social systems analysis, or other rigorous research methodologies to evaluate impact and identify potential harm to users as well as broader groups of people.",,,No monitoring protocols have been established: Necessary infrastructure needed to perform monitoring of AI systems in production is not available and/or a plan to monitor models has not yet been established.,No - Some individual decisions or actions require direct human oversight.,,Other ,Only internal employees use the result of this use case. They are trained on the use of the results.,No,N/A,Other,Employee testing and review of predicted labels,Yes,,Other,A staff member can overrule the predicted label Patents - Skill Group Matching,Department of Commerce,DOC,USPTO - United States Patent and Trade Office,Mission-Enabling,,None of the above.,AI will permit adoption of Skill Groups system for delineating subject matter of patent applications by automating the allocation of Skill Group designations to applications. Benefits: Cost reduction from eliminating dependence on outdated US Patent Classification (USPC) system; cost savings from reducing Skill Group designation workload; improvement in availability of metrics data; improvements in ability to manage unexamined patent inventory and workforce allocation/hiring,"Prediction of the correct Skill Group describing the technology/subject matter claimed in a patent application, along with a confidence score indicating the likelihood of each possible Skill Group.",Acquisition and/or Development,Neither,2022-08 (estimated),2022-09-13 Technical kickoff (no new acquisition - supported by existing contract),,,Developed with both contracting and in-house resources.,280-EIPL-0762 (OY2),Yes,,,No,,No,,Yes,,Full text of published patent documents; USPC classifications thereof; USPC-to-Skill Group correspondence table,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",Other,Does not use any demographic variables.,,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Groundwater Modeling,Department of Energy,DOE,LM HQ - Office of Legacy Management (LM),Science & Space,,Creating visual representations of data sets for reports and presentations using AI.,Predicts flow of ground water containments underground.,Parameter estimation for return to groundwater models,Operation and Maintenance,Neither,1/1/2003,2/1/2003,10/1/2003,,Developed with both contracting and in-house resources., IDIQ Contract # 89303020DLM000001.,No,,,Yes,Parameters and model outputs are reviewed and validated by a subject matter expert,No,No,Other,"Data for this use case are stored in a single repository, but it is not yet enterprise-wide data.",LM maintains historically collected groundwater data to train the model.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case's development.",Other,,No,"Yes – agency has access to source code, but it is not public.",https://pesthomepage.org/programs,Yes,Legacy Management General Support System,,,,,,,,,,,,,,,,,,,,,,,,,,, OpenText for Records Management (Email Auto-Classification),Department of Energy,DOE,EE HQ - EE Headquarters (EE),Mission-Enabling (internal agency support) ,,None of the above.,Reduce time required to classify email records.,OpenText will categorize each email message to an appropriate category.,Retired,Neither,,,,6/1/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Applications of Natural Language Processing and Similarity Measures for Similarity Ranking,Department of Energy,DOE,EHSS HQ - Office of Environment Health Safety & Security (EHSS),Other,,None of the above.,"(1) Demonstrate Capabilities of AI (2) Cost savings by sharing algorithms, approaches and lessons learned of those evaluated and deployed in the ES&H DAMaL Tools.","Custom output (e.g., ranking of records)",Operation and Maintenance,Neither,10/1/2021,10/1/2021,10/1/2021,,Developed in-house.,N/A,No,,,No,N/A,No,No,No,,Approach is NOT supervised ML.,"Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",Other,,Yes,"Yes – agency has access to source code, but it is not public.",Code is based on open source approaches. Link to open source code is not provided.,No,N/A,Less than 6 months,No,,Yes,,Other,"All algorithms, code, training, is developed and tested in a desktop and Python environment and if successful then deployed under a contract managed MS Azure environment.",None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,No,No testing: No testing of the model to simulate performance in an operational environment has been conducted thus far.,N/A,Planned or in-progress,,,,,N/A,No,N/A,,N/A,No – it is not operationally practical to offer this.,System runs locally and run on a desktop in a local Python enviornment.,Other,System runs locally and run on a desktop in a local Python enviornment. "Data Analytics and Machine Learning (DAMaL) Tools to enhance the analysis of Environment, Safety and Health (ES&H) data: Classification, Robotic Process Automation and Data Visualization",Department of Energy,DOE,EHSS HQ - Office of Environment Health Safety & Security (EHSS),Other,,None of the above.,"(1) Data Visualization using AI (2) Improve efficiency of obtaining insights from data (e.g., ES&H data)",Dynamic data visualization charts,Operation and Maintenance,Safety-impacting,8/1/2018,8/1/2018,8/1/2018,,Developed with both contracting and in-house resources.,"GSA Contract #: GS-00F-0004T Contract ID # Support to EHSS: 89243321FAU400002",No,,,No,N/A,No,Yes,No,,"Utilizes ORPS data for training and applies labels to non-ORPS ES&H Data (e.g., CAIRS, FPRS, DOE OPEXShare, etc.)","Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",Other,,Yes,"Yes – agency has access to source code, but it is not public.",Code is based on open source approaches but link was not provided.,Yes,Reporting and Analytical System (RAS),Less than 6 months,No,,Yes,,Other,"All algorithms, code, training, is developed and tested in a desktop and Python environment and if successful then deployed under a contract managed MS Azure environment.",None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,Yes,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.",Key risk from using the AI is providing misleading information to ES&H practitioners and subject matter experts.,Planned or in-progress,"Automated and Regularly Scheduled Updates: Some aspects of the plan to monitor the AI system post-deployment are automated including re-training of models after detecting drift; however, data science teams are still significantly involved in the monitoring and re-deployment process.",No - Some individual decisions or actions require direct human oversight.,,Email ,"Tools available to all DOE Employees but require direct request to the Development team. Various communications (e.g., welcome email, tools login landing page, demos) and documentation (e.g., users guide) include extensive details on how the tools use AI as part of its dashboards.",No,N/A,,"The system supports and affect DOE employees that use the system. The ES&H DAMaL Tools have various processes in place to consult and incorporate feed back (e.g., email, demos, training, user testing, feedback form, etc.).",Yes,,Yes, "Data Analytics and Machine Learning (DAMaL) Tools for Analysis of Environment, Safety and Health (ES&H) data: Similarity Based Information Retrieval",Department of Energy,DOE,EHSS HQ - Office of Environment Health Safety & Security (EHSS),Other,,None of the above.,"(1) Searching for Information using AI (2) Improve efficiency of finding relevant data (e.g., ES&H data)",Ranking of records,Operation and Maintenance,Safety-impacting,10/1/2021,8/1/2018,10/1/2021,,Developed with both contracting and in-house resources.,"GSA Contract #: GS-00F-0004T Contract ID # Support to EHSS: 89243321FAU400002",No,,,No,N/A,No,Yes,No,,"ES&H Data (e.g., ORPS, CAIRS, FPRS, DOE OPEXShare). Approach utilized is NOT supervised ML.","Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",Other,,Yes,"Yes – agency has access to source code, but it is not public.",Code is based on open source approaches. Open source link not provided.,Yes,Reporting and Analytical System (RAS),Less than 6 months,No,,Yes,,Other,"All algorithms, code, training, is developed and tested in a desktop and Python environment and if successful then deployed under a contract managed MS Azure environment.",None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,Yes,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.","Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.",Planned or in-progress,"Automated and Regularly Scheduled Updates: Some aspects of the plan to monitor the AI system post-deployment are automated including re-training of models after detecting drift; however, data science teams are still significantly involved in the monitoring and re-deployment process.",No - Some individual decisions or actions require direct human oversight.,,Email ,"Tools available to all DOE Employees but require direct request to the Development team. Various communications (e.g., welcome email, tools login landing page, demos) and documentation (e.g., users guide) include extensive details on how the tools use AI as part of its dashboards.",No,N/A,,N/A,Yes,,Yes, "Data Analytics and Machine Learning (DAMaL) Tools to enhance the analysis of Environment, Safety and Health (ES&H) data: Unsupervised Machine Learning Text Clustering",Department of Energy,DOE,EHSS HQ - Office of Environment Health Safety & Security (EHSS),Other,,None of the above.,"(1) Grouping Reports using AI (i.e., Clustering) (2) Improve efficiency of grouping and highlighting important data (e.g., ES&H data)",Clustering of records with top terms and topics,Operation and Maintenance,Safety-impacting,8/1/2018,8/1/2018,8/1/2018,,Developed with both contracting and in-house resources.,"GSA Contract #: GS-00F-0004T Contract ID # Support to EHSS: 89243321FAU400002",No,,,No,N/A,No,Yes,No,,"ES&H Data (e.g., ORPS, CAIRS, FPRS, DOE OPEXShare). Approach utilized is Unsupervised ML","Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",Other,,Yes,"Yes – agency has access to source code, but it is not public.",Code is based on open source approaches but open source code link not provided,Yes,Reporting and Analytical System (RAS),Less than 6 months,No,,Yes,,Other,"All algorithms, code, training, is developed and tested in a desktop and Python environment and if successful then deployed under a contract managed MS Azure environment.",None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,Yes,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.",Key risk from using the AI is providing misleading information to ES&H practitioners and subject matter experts.,Planned or in-progress,"Automated and Regularly Scheduled Updates: Some aspects of the plan to monitor the AI system post-deployment are automated including re-training of models after detecting drift; however, data science teams are still significantly involved in the monitoring and re-deployment process.",No - Some individual decisions or actions require direct human oversight.,,Email ,"Tools available to all DOE Employees but require direct request to the Development team. Various communications (e.g., welcome email, tools login landing page, demos) and documentation (e.g., users guide) include extensive details on how the tools use AI as part of its dashboards.",No,N/A,,"The system supports and affect DOE employees that use the system. The ES&H DAMaL Tools have various processes in place to consult and incorporate feed back (e.g., email, demos, training, user testing, feedback form, etc.).",Yes,,Yes, Fast AI for real-time/online diagnostics and calibration; AI-based autonomous system control/operation optimization (including anomaly/failure detection),Department of Energy,DOE,SLAC - SLAC National Accelerator Laboratory (SC43 OIM),Science & Space,,None of the above.,"AI/ML to improve efficiency and quality of accelerator-based scientific user facility operation, including for light sources. Benefits include higher science throughput in scientific user facilities, greater energy efficiency, new scientific capabilities at Office of Science user facilities. ","Prediction of acclerator beam behavior, decisions about controllable variables to adjust to improve beam behavior. ",Implementation and Assessment,Neither,10/1/2019,10/1/2019,9/30/2023,,Developed in-house.,,No,,,No,,No,No,No,,"Training data are collected from physics simulations and the accelerator as it runs. ","Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",Other,,Yes,Yes – source code is publicly available.,https://github.com/xopt-org,No,,Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, "AI for system design optimization (e.g., detector, accelerator)",Department of Energy,DOE,SLAC - SLAC National Accelerator Laboratory (SC43 OIM),Science & Space,,None of the above.,"AI/ML models a relatively small data sets in a high dimensional parameter space and provides accurate predictions of future data points. The prediction capability in turn improves the convergence efficiency of the optimization algorithm, which is used for time consuming design optimization. ","The AI system is a non-parametric data fitting model and its output is predicted function values of any new solution in the parameter space and the uncertainties. ",Operation and Maintenance,Neither,10/1/2017,10/1/2017,9/30/2020,,Developed in-house.,,No,,,No,,No,No,No,,"Training data are collected at the time of running the optimization algorithm. ","Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",Other,,No,Yes – source code is publicly available.,https://github.com/SPEAR3-ML/MG-GPO,No,,Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, PARSGPT,Department of Energy,DOE,PM HQ - Office of Project Management (PM),Mission-Enabling (internal agency support) ,,None of the above.,The value-add is derived from providing an accessible way for PM Analysts to safely interact with LLM technology.,Free form text response to questions (Chatbot),Operation and Maintenance,Neither,8/18/2023,8/23/2023,9/10/2023,,Developed in-house.,,No,,,No,,No,No,Yes,,NA,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",Other,,Yes,"Yes – agency has access to source code, but it is not public.",NA,Yes,Project Assessment and Reporting System (PARS),Less than 6 months,Other,Not applicable. ,Other,Not applicable.,Other,Not applicable.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, CrowdStrike,Department of Energy,DOE,LM HQ - Office of Legacy Management (LM),Other,,Identifying unusual patterns in system logs from a single incident report using AI.,"Predicts malicious behavior, provides decisions and blocks malicious behavior.","Prediction, Recommendation, Decision",Operation and Maintenance,Neither,6/15/2023,6/15/2023,6/15/2023,,Developed in-house.,,No,,,No,Not Applicable,No,No,No,,LMs client and server logs data set,"Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",,,No,No – agency does not have access to source code.,,Yes,Legacy Management General Support System,Less than 6 months,Other,We are using pre-existing LM resources.,No,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,No,No testing: No testing of the model to simulate performance in an operational environment has been conducted thus far.,Privacy Loss,Planned or in-progress,No monitoring protocols have been established: Necessary infrastructure needed to perform monitoring of AI systems in production is not available and/or a plan to monitor models has not yet been established.,Yes - All individual decisions or actions are automated and may be rendered without direct human interaction.,,N/A - individuals are not interacting with the AI for this use case,,No,Manual Review,None of the above,,No – it is not operationally practical to offer this.,AI use case does not impact public,"No – This AI use case is not subject to the opt-out requirement because the AI functionality is solely used for the prevention, detection, and investigation of fraud or cybersecurity incidents, or the conduct of a criminal investigation.", SCMC AWS Data Infrastructure,Department of Energy,DOE,KCNSC - Kansas City National Security Campus (KCFO),Mission-Enabling (internal agency support) ,,Creating visual representations of data sets for reports and presentations using AI.,Automation of multiple manual processes and improvement of data quality and governance.,Initially the system will use M.L. to identify inconsistent data and make preditions of how that data should be organized. ,Implementation and Assessment,Neither,5/1/2024,8/1/2024,,,Developed with contracting resources.,Procured under SCMC Agreement # WS181016,No,,,No,,No,No,Yes,,Non production related procurement data is currently being collected and managed by the SCMC. This is the data set that wil be used by the system.,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups.",,,No,No – agency does not have access to source code.,,Yes,AWS US EAST/WEST ,Less than 6 months,No,,No,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,No,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.",Not applicable to the risks defined above,Planned or in-progress,"Automated and Regularly Scheduled Updates: Some aspects of the plan to monitor the AI system post-deployment are automated including re-training of models after detecting drift; however, data science teams are still significantly involved in the monitoring and re-deployment process.",No - Some individual decisions or actions require direct human oversight.,,"Other, if other please explain in next question.",Not applicable to the definitions defined above,No,Not applicable to the scenario defined above,Other,Not applicable to the scenario defined above,"No – Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.",There is no AI system outcome that would have an impact on any specific individual or group,Yes, ServiceNow Predictive Intelligence,Department of Energy,DOE,LANL - Los Alamos National Lab (LAFO),Mission-Enabling (internal agency support) ,,Prioritizing and categorizing incoming emails using AI.,"Provide better and more consistent classification of ticket data entered into ServiceNow ","Field classification data ",Operation and Maintenance,Neither,10/1/2023,10/1/2023,12/1/2023,,Developed in-house.,,No,,,No,,No,No,No,,"Existing ticket data is used to train the model with data and training servers stored within FedRAMP High data centers where ServiceNow is hosted ","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups.",Other,,No,No – agency does not have access to source code.,N/A,Yes,ServiceNow ATO,Less than 6 months,Yes,,No,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,"Automated and Regularly Scheduled Updates: Some aspects of the plan to monitor the AI system post-deployment are automated including re-training of models after detecting drift; however, data science teams are still significantly involved in the monitoring and re-deployment process.",No - Some individual decisions or actions require direct human oversight.,,,,No,,,,Yes,,Yes, AI-Enhanced Lab Assist,Department of Energy,DOE,PNNL - Pacific Northwest National Laboratory (SC43 OIM),Mission-Enabling (internal agency support) ,,Logging and analyzing time spent on tasks using AI-powered time management tools.,"Integrating lessons learned into Lab Assist Activity Planning to enhance operational efficiency, improve information sharing leveraging best practices, and foster a culture of continuous improvement. ",Leverage AI for trend detection in working planning and control data.,Initiated,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Microsoft 365 Copilot (Productivity Suite),Department of Energy,DOE,PNNL - Pacific Northwest National Laboratory (SC43 OIM),Mission-Enabling (internal agency support) ,,Improving the quality of written communications using AI tools.,"This helps PNNL streamline workflows, improve efficiency, and allow researchers to focus more on innovation and less on administrative tasks, ultimately accelerating scientific research and operational effectiveness.","Using Microsoft 365 Copilot, PNNL aims to produce enhanced document quality, increased efficiency, insightful data analysis, improved collaboration, automated workflows.",Initiated,Neither,10/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, NLCOO AI for Lessons Learned tool ,Department of Energy,DOE,LANL - Los Alamos National Lab (LAFO),Mission-Enabling (internal agency support) ,,Searching for information using AI.,"Better search to gain insights from exisitng lessons learned to improve how we do work. ","Search list of relevant documents ",Operation and Maintenance,Neither,2/27/2024,3/16/2024,3/16/2024,,Developed in-house.,,No,,,No,,No,No,No,,"Not trained on any agency or LANL data ","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case's development.",Other,,Yes,Yes – source code is publicly available.,N/A,Yes,"Authorization to Operate Amazon Web Service Federal Risk and Authorization Management Program Infrastructure as a Service Cloud Service Provider, Amazon Web Service Information System Security Plan, SD-006-CP-002/L2",Less than 6 months,No,,Yes,,No,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,,No – Agency did not request an extension for this use case.,,,,,No monitoring protocols have been established: Necessary infrastructure needed to perform monitoring of AI systems in production is not available and/or a plan to monitor models has not yet been established.,No - Some individual decisions or actions require direct human oversight.,,,,No,,,,Yes,,Yes, " Spot for automated sensing, inspection, and capture",Department of Energy,DOE,PNNL - Pacific Northwest National Laboratory (SC43 OIM),Science & Space,,None of the above.,"Spot, a mobile robot, can navigate hazardous or hard-to-reach areas to perform inspections and gather precise data with its advanced sensors and cameras.","Improving efficiency, and the accuracy of sensor research data.",Initiated,Neither,10/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Microsoft Power Platform capability that provides AI to automate processes in Power Apps and Power Automate,Department of Energy,DOE,PNNL - Pacific Northwest National Laboratory (SC43 OIM),Other,,Creating visual representations of data sets for reports and presentations using AI.,"By leveraging AI for automation, PNNL can automate routine tasks such as data entry, reporting, and workflow management, freeing up researchers and staff to focus on higher-value activities. ","Enhances productivity, reduces human error, and leads to more efficient management of research projects and resources.",Initiated,Neither,10/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, CAISY,Department of Energy,DOE,HC HQ - Office of the Chief Human Capital Officer (HC),Education & Workforce,,None of the above.,"1. CAISY provides interactive, scenario-based content powered by AI where a learner can practice new skills in a safe space. This AI simulation content allows learners to choose a role, practice specific skills by responding to AI prompts, and receive adaptive, personalized feedback to guide their development 2. The user is introduced by an avatar that is generated with AI text-to-video. After the introduction, the learner can either interact by typing or using speech-to-text (STT) and text-to-speech (TTS) services for more immersive and natural interaction. When the conversation is over, the learner will receive a rating and evaluation ",speech-to-text (STT) and text-to-speech (TTS) services for more immersive and natural interaction.,Operation and Maintenance,Neither,1/4/2024,8/21/2024,8/21/2024,,Developed with contracting resources.,N/A,No,,,No,,No,No,No,,None of the above,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups.",Other,,No,No – agency does not have access to source code.,,Yes,PERCIPIO/CAISY,Less than 6 months,No,,No,,No,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, ServiceNow Virtual Agent,Department of Energy,DOE,LANL - Los Alamos National Lab (LAFO),Mission-Enabling (internal agency support) ,,Searching for information using AI.,"Provide troubleshooting assistance via chat to employees seeking assistance on IT-related issues ","Chat responses ",Operation and Maintenance,Neither,10/1/2023,1/1/2024,6/1/2024,,Developed in-house.,,No,,,No,,No,No,No,,"Existing ticket data is used to train the model with data and training servers stored within FedRAMP High data centers where ServiceNow is hosted ","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups.",Other,,,No – agency does not have access to source code.,N/A,Yes,ServiceNow ATO,Less than 6 months,Yes,,No,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,,No – Agency did not request an extension for this use case.,,,,,"Automated and Regularly Scheduled Updates: Some aspects of the plan to monitor the AI system post-deployment are automated including re-training of models after detecting drift; however, data science teams are still significantly involved in the monitoring and re-deployment process.",No - Some individual decisions or actions require direct human oversight.,,,,No,,,,Yes,,Yes, Databricks AI for Cloud Data Warehouse,Department of Energy,DOE,PNNL - Pacific Northwest National Laboratory (SC43 OIM),Other,,None of the above.,Efficiency in analytics and deployment of AL/ML models,Recommendation based on analytic input ,Initiated,Neither,10/1/2024,10/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, DOE Technical Standards,Department of Energy,DOE,EHSS HQ - Office of Environment Health Safety & Security (EHSS),Mission-Enabling (internal agency support) ,,Improving the quality of written communications using AI tools.,"Enhance quality and accuracy of technical standards, supporting DOE's commitment to safety excellence. ",Recommendations and feedback for improvement,Operation and Maintenance,Neither,,,,,,,No,,,,Organizational-level business workflows integrates quality assurance procedures conducted by subject matter experts (SMEs) for all proposed responses and their publication(s).,No,,No,,N/A,,,,No,No – agency does not have access to source code.,Commercial Chatbots are used.No link provided.,No,N/A,,Other,N/A,Other,N/A,Other,N/A,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,No,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.","Standards must align with regulatory requirements, which are subject to updates that an AI chatbot might not capture effectively. AI chatbot might misinterpret technical terminology or regulatory requirements, requiring human review. ",,,,,,Reasonable & Timely Notice that applies: In Person (used by program staff),No,N/A,,N/A,No – it is not operationally practical to offer this.,N/A,Other,The Chatbot(s) can alternately be used by the program team since they areoperated stand-alone. Microsoft Copilot for Security,Department of Energy,DOE,PNNL - Pacific Northwest National Laboratory (SC43 OIM),Mission-Enabling (internal agency support) ,,Identifying unusual patterns in system logs from a single incident report using AI.,"Proactively identify, mitigate, and respond to threats. Copilot assists in real-time monitoring, threat detection, incident response automation, and vulnerability management.","The intended outputs of using Microsoft Copilot for Security at PNNL include real-time threat detection, automated incident response, enhanced data protection, compliance reports, security insights, reduced cyber risk, and sustained operational continuity",Initiated,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Text To Speech Audio Generation,Department of Energy,DOE,KCNSC - Kansas City National Security Campus (KCFO),Education & Workforce,,Improving the quality of written communications using AI tools.,AI tool is used to generate audio files to be used in training content.Allows for faster development of training to meet company needs. This leads to a cost savings in both initial development and future modifications. ,MP4 Audio Files,Operation and Maintenance,Neither,,,9/16/2022,,,,No,,,No,,No,No,No,,"We are not training the AI model, project is COTS",,,,No,No – agency does not have access to source code.,,Yes,"N/A - System is used only for GREEN DATA, no OUO or CUI is allowed and does not require a System Security Plan to be named",Less than 6 months,No,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,,No – Agency did not request an extension for this use case.,No,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.",No Risks Identified,Planned or in-progress,No monitoring protocols have been established: Necessary infrastructure needed to perform monitoring of AI systems in production is not available and/or a plan to monitor models has not yet been established.,No - Some individual decisions or actions require direct human oversight.,,,,No,,Post transaction customer feedback collections,,Yes,,Yes, Lex Natural Language Interface ,Department of Energy,DOE,NREL - National Renewable Energy Laboratory (EE),Mission-Enabling (internal agency support) ,,Collaborating in real-time using AI-assisted tools in word processors.,"Create an efficient and user-friendly system that enables users to query project data, such as funding, AUs (allocation units), project focus, and fiscal years, with natural language prompts. ","The system executes a query against a postgres database and displays an LLM-generated textual summary of returned query records. The system also displays the LLM generated queries. ",Acquisition and/or Development,Neither,9/25/2024,10/1/2024,11/1/2024,,Developed in-house.,NA,No,,,No,,No,No,Yes,,"Structured data from a PostgreSQL database containing information about HPC (High-Performance Computing) projects. ","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case's development.",,,Yes,"Yes – agency has access to source code, but it is not public.",N/A,Yes,"NREL Stratus AWS Compute Environment, NREL Azure System",Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,Planned or in-progress.,"Benchmark evaluation: Testing of the AI model has involved the use of benchmarks (either publicly available or internally created) to estimate performance in real-world settings, but has not been tested in an operational environment. ","Misinterpretation of data due to model errors could lead to missed opportunities for users in receiving accurate project information. Inaccurate query results or inefficient use of resources may lead to financial costs associated with misinformed decision-making. ",Planned or in-progress,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,Online – in the terms or instruction for the service,"Online - Site will have user terms of service persuant to Section 5(c)(iv)(I) of OMB Memorandum M-24-10. ",No,"Process to detect bias and fairness are in testing and being regulary evaluated by model provideers. There are built-in guardrails in place to prevent abuse ",None of the above,,Yes,,Yes, Scopus AI,Department of Energy,DOE,NE INL - NE Idaho National Laboratory (NE),Science & Space,,None of the above.,Assist researchers by reducing time to find applicable research while increasing quality and accuracy of identified hits.,"Research citations, abstraacts and other summaries.",Operation and Maintenance,Neither,7/1/2024,8/6/2024,8/6/2024,,Developed with contracting resources.,"Scopus AI was purcahsed, added as part of the Elsevier/Scopus subcontract.",No,,,No,,No,No,No,,"None. Scopus AI uses publicly available journal abstracts, no agency data is used.","Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",N/A,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,"Developed by Elsevier/Scopus with data publicly, but available journal articles. Some journals are copywritten.",Other,We will not be training the model.,Other,Not applicable.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,No,No testing: No testing of the model to simulate performance in an operational environment has been conducted thus far.,"Key functional risks include fabricated & inaccurate answers as well as model & output bias. Mitigated operational risks include data privacy, fraud and exploit risks.",Yes – by another appropriate agency office that was not directly involved in the system’s development,"Automated and Regularly Scheduled Updates: Some aspects of the plan to monitor the AI system post-deployment are automated including re-training of models after detecting drift; however, data science teams are still significantly involved in the monitoring and re-deployment process.",No - Some individual decisions or actions require direct human oversight.,,In person ,,No,"Not using PII, commercial solution",None of the above,,No – it is not operationally practical to offer this.,This is an internally used use case. Primary feedback mechanisms are direct use testing and customer feedback.,Yes, EnerGPT,Department of Energy,DOE,IM-50 - Architecture Engineering Technology and Innovation (IM),Mission-Enabling (internal agency support) ,,None of the above.,EnerGPT aims to enhance user productivity and reduce time spent on redundant tasks . ,EnerGPT generates answers to user questions. ,Acquisition and/or Development,Neither,7/8/2024,7/18/2024,,,Developed with both contracting and in-house resources.,89303019AIM000005,No,,,,,,,No,,"Publicly available government documentation was used to evaluate the model, an example document is the Administration Cybersecurity Priorities for the FY 2026 Budget PDF (https://www.whitehouse.gov/wp-content/uploads/2024/07/FY26-Cybersecurity-Priorities-Memo_Signed.pdf) published by the White House. ","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case's development.",N/A,,,,,Yes,OCIO Google Cloud Platform,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, MAPPRITE,Department of Energy,DOE,EHSS HQ - Office of Environment Health Safety & Security (EHSS),Mission-Enabling (internal agency support) ,,Searching for information using AI.,"(1) The implemented AI will help automate data mining, ingesting, and indexing of existing disparate organizational data sources information for relevant safeguards and security (S&S) support information; and the (2) expected benefits will be to help improve EHSS-51 business workflows for researching potentially relevant S&S support data available such that the information will be accessible and searchable by policy subject matter specialists for awareness and additional context for strategic decision-making and policy management ","In its full implementation phase, the application's AI output will provide S&S policy [support] data available such that the information will be accessible and searchable by policy subject matter specialists for strategic decision-making and policy management without having to manually search through hundreds of sources for relevant information.",Acquisition and/or Development,Neither,6/22/2023,7/15/2024,,,Developed with both contracting and in-house resources.,Leveraged existing contract vehicle with DOE laboratory,No,,,No,"The data for this application is being acquired from multiple existing sources (laws, regulations, standards) and from knowledge possessed by subject matter experts. Data review and curation processes are being developed and applied.",Yes,No,No,,"Department of Energy Directives. Requirement source documents, such as statutes, regulations and standards were also provided to the development team to assist with ingesting content to the AI model via AWS Kendra. ","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case's development.",,,Yes,"Yes – agency has access to source code, but it is not public.",Code is not publicly available,Yes,Provisional Authorization to Operate (P-ATO) for the Methodology for Analyzing and Prioritizing Policy Requirements and Integrating Them for Effectiveness (MAPPRITE),6-12 months,Other,The required IT infrastructure was provisioned and coordinated via the hosting lab's intake process. This application will also leverage EHSS-72 change control process.,Other,Computing resources for the current development and testing phases were coordinated via the lab hosting the application software prototype. The application will be on a cloud-service provider (AWS GovCloud). ,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, MI8 Collimators Surogate Model,Department of Energy,DOE,FNAL - Fermi National Accelerator (SC43 OIM),Science & Space,,None of the above.,This project is working to create a ML surogate model of the exisitng MI8 collimation system. The purpose of the ML model is to aid in the tuning of the collimation system for acclerator operations and to help find more optimal settings in a timely manner. We hope to extend these techniques to other sub-systems and also a new MI8 collimation system being installed.,The ML outputs of the system are predictions of collimation system performance given collimation system settings and beam charecteristics.,Acquisition and/or Development,Neither,10/1/2023,10/1/2024,10/1/2024,,Developed in-house.,,No,,,No,,No,No,No,,Accelerator operations machine data,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case's development.",Other,,Yes,Yes – source code is publicly available.,https://github.com/fermi-mi/collimators-ml,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,N/A,,,,None of the above,,,,, Records Digitization,Department of Energy,DOE,SEPA - Southeastern Power Administration (PMA),Mission-Enabling (internal agency support) ,,Digitizing text from scanned documents or smart forms for archival purposes using AI.,SEPA is leveraging AI and ML to enhance metadata tagging and quality control during the records digitization process.,Acurate metadata assignment in accordance with SEPA's NARA-approved file plan/records schedule.,Acquisition and/or Development,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Machine Learning components within Splunk Enterprise Security,Department of Energy,DOE,SLAC - SLAC National Accelerator Laboratory (SC43 OIM),Mission-Enabling (internal agency support) ,,Identifying unusual patterns in system logs from a single incident report using AI.,"Allows for big data analysis and identification of trends in large datasets. SLAC expects to be able to perform data clustering, trend analysis, classification, and regression by using Splunk ","Prediction and clustering models ",Operation and Maintenance,Neither,8/1/2019,8/1/2019,9/1/2020,,Developed with both contracting and in-house resources.,SLAC-0000238366,No,,,No,,No,No,No,,"Splunk's machine learning model is trained on data generated by activity at SLAC ","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups.",Other,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Machine Learning components within CrowdStrike,Department of Energy,DOE,SLAC - SLAC National Accelerator Laboratory (SC43 OIM),Mission-Enabling (internal agency support) ,,Identifying unusual patterns in system logs from a single incident report using AI.,"CrowdStrike uses machine learning to review security events in order to create notifications of detections and incidents for the SLAC Cybersecurity team. From the use of CrowdStrike's machine learning components, SLAC receives the benefit of visibility to analyze possible security events ","CrowdStrike Detections or Incidents ",Operation and Maintenance,Neither,1/27/2021,1/27/2021,7/7/2021,,Developed with contracting resources.,"SLAC-0000208966 ",No,,,No,,No,No,No,,"CrowdStrike's machine learning model is trained on data generated by activity at SLAC ","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups.",,,No,No – agency does not have access to source code.,,Yes,CrowdStrike Falcon,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, AskOEDI,Department of Energy,DOE,NREL - National Renewable Energy Laboratory (EE),Mission-Enabling (internal agency support) ,,Collaborating in real-time using AI-assisted tools in word processors.,Making data more accessible and user-friendly for the public. ,The systems leverages Retrieval-Augmented Generation to find semantically relevent content which the AI (LLM) summarizes for the end user as a method to describe relevent content within the data catalog. ,Operation and Maintenance,Neither,9/30/2024,10/1/2024,10/30/2024,,Developed in-house.,NA,No,,,Yes,"To demonstrate compliance with the DOE Information Quality Act (IQA) guidelines, the approach with RAG and LLM-based applications ensures the quality, objectivity, utility, and integrity of disseminated information. Quality and Objectivity: The approach uses a locked-down Retrieval-Augmented Generation (RAG) system that minimizes inaccuracies (hallucinations) and excludes memory components, ensuring accurate, clear, and reliable information presentation. This satisfies the IQA's requirement for objective, unbiased dissemination and maintains transparency. Utility: Integrating metadata-driven semantic search in the vector database enhances the utility of DOE's data catalog for public users, meeting the IQA's standard for usefulness. This utility-focused design ensures that data is accessible, relevant, and usable for public insights, supporting transparency and public knowledge. Transparency and Reproducibility: The approach supports data reproducibility by basing responses on catalog metadata and summary information, ensuring consistency and enabling independent reanalysis by qualified users. Transparency is further upheld through documentation of the RAG interactions, aligning with the IQA's standards for open access and transparency in information dissemination. Response to Public Needs: This design incorporates continuous feedback to improve relevance and usability, engaging public and stakeholder needs effectively. It aligns with the IQA's goal of considering public feedback in information quality and ensuring accountability through documented, high-level review processes for data corrections if requested. By adhering to these DOE IQA guidelines, this approach ensures public trust in DOE's open data, demonstrating alignment with the standards of quality, utility, objectivity, and integrity as required under OMB M-19-15.",No,No,Yes,,As this is summarizing data from the OEDI data repository the related datasets are described by the catalog which is also used to validate the AI responses: https://data.openei.org/,"Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",,,Yes,"Yes – agency has access to source code, but it is not public.",N/A,Yes,NREL Stratus AWS Compute Environment and NREL Azure System,Less than 6 months,Yes,,No,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,,No – Agency did not request an extension for this use case.,Planned or in-progress.,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.","As the use case for the AI has been limited to summarization of existing metadata the key risks were identified as LLM hallucinations or LLM generated text which would cause reputational harm to DOE. Techniques such as red-teaming, adversarial testing, and prompt-injection testing were used to evaluate constraint of hallucinations and LLM. ",Planned or in-progress,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,Online – in the terms or instruction for the service,,No,Process to detect bias and fairness are in testing and being regulary evaluated by model provideers. There are built-in guardrails in place to prevent abuse.,None of the above,,No – it is not operationally practical to offer this.,"As the action of the system is to summarize and communicate existing information returned for a query, the ability of the AI to make impact individuals or groups is significantly limited. ",Other,Other methods of searching content using common search methodlogies are also available to the catalog users. Hanford Search,Department of Energy,DOE,RL - Hanford - Richland Operations Office - Hanford (EM),Mission-Enabling (internal agency support) ,,Searching for information using AI.,"The purpose of the Hanford Search is to provide similar functionality to our Hanford Search application without needing multiple applications. The benefits of the AI are that there is a single interface with multiple uses and the AI can provide better, more relevant search results. Additionally, users can ask questions in natural language instead of needing to input specific search criteria. ","The system outputs text respones from user prompts requesting information on grounded data related to Hanford Search Index ",Acquisition and/or Development,Neither,6/24/2024,6/23/2024,,,Developed with contracting resources.,"89303320DEM000031 ",No,,,No,,No,Yes,Yes,,"Our Data does not train the models. ","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case's development.",N/A,,Yes,No – agency does not have access to source code.,,Yes,Microsoft Azure OpenAI,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,No,No testing: No testing of the model to simulate performance in an operational environment has been conducted thus far.,"Risks identified in this AI were incorrect or halucination responses from the AI. Users are trained and informed to validate the AI responses before use. They were identified through discimination of information of risk from other ChatGPT applications along with developer and user testing. ",Planned or in-progress,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,"Other, if other please explain in next question.","HMIS is collaborating with, and recieiving approval from, our local DOE on our internal communication plans. Once our internal developement is ready for customer deployment and communications, HMIS will provided their use cases to all users via an AI website where users can view and request use cases. When a use case gets developed, those effected will be communication with about the AI, and provided informational material, training, and feedback loops to provide reasonable and timely notices. Reasonable and Timely notices include Email, Team Channel, and Online. ",No,"The AI does not take into account any demographic information when generating responses, and does not use any demographic information about employees specifically to generate results. Our AI use case should have have any demographic disparity. If a users were to find an issue with a response, they can provide feedback via a feedback mechanism with each response and provide a summary of their feedback. Developers would then address the issue and look to mitigate any issues found. ",Other,"The development of our AI was done with the consultation of our affected community, where affected community is the community of users who would be using the AI, as well as the community of DOE sites that have employed similar AI tools. We have consulted individually and in groups our community and provided early demonstration, feedback mechanisms, direct user testing, post-transaction feedback collection. Areas (Team Channels) where listening and meetings on topics can occur will be provided before release. ",No – it is not operationally practical to offer this.,"There currently is no escalation process fro the AI use case to evaluate the impact on the individual or group. Use of the AI is a productivty use and is voluntary to the users. Users who do not want to use the AI are not required and any issues personally that arise with an individual or group related to the AI can be captured via our feedback, help, and AI Team communication channels and will be reviewed by IMS Leadership. ",Yes, AI Chat Bot for IT User Services,Department of Energy,DOE,PPPL - Princeton Plasma Physics Laboratory (SC43 OIM),Mission-Enabling (internal agency support) ,,None of the above.,"Interactive RAG chat bot model trained on existing, updated and new IT resource documentation in the user space. Platform will be used as an informative method for users to handle tier 1 IT issues and help guide to the correct place.",AI output will be recommendations and instructions based on training data from IT administrators in more user friendly responses,Initiated,Neither,5/1/2024,9/1/2024,,,Developed in-house.,,No,,,No,,No,No,No,,Help Desk Knowledge Base Article and other supporting documentation in the user space,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case's development.",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,,No – Agency did not request an extension for this use case.,Planned or in-progress.,No testing: No testing of the model to simulate performance in an operational environment has been conducted thus far.,No key risks identified,Planned or in-progress,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,,Email,No,"RAG Model will be significantly grounded in factual data, limiting creativity in responses",,Independent feedback links will be given to receive feedback data outside of the model,Yes,,Yes, Hanford Popfon Search,Department of Energy,DOE,RL - Hanford - Richland Operations Office - Hanford (EM),Mission-Enabling (internal agency support) ,,Searching for information using AI.,"The purpose of the Hanford Popfon Search is to provide similar functionality to our employee look-up application without needing multiple applications. The benefits of the AI are that there is a single interface with multiple uses and the previous application, which is older in architecture, can be retired, providing a safer, more secure, and cost effective alternative. Additionally, users can ask questions in natural language instead of needing to input specific search criteria. ","The system outputs text respones from user prompts requesting information on grounded data related to employee contact and organization information ",Acquisition and/or Development,Neither,5/7/2024,6/22/2024,,,Developed with contracting resources.,89303320DEM000031,No,,,No,,No,Yes,Yes,,Our Data does not train the models.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case's development.",N/A,,Yes,No – agency does not have access to source code.,,Yes,Microsoft Azure OpenAI,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,No,No testing: No testing of the model to simulate performance in an operational environment has been conducted thus far.,Risks identified in this AI were incorrect or halucination responses from the AI. Users are trained and informed to validate the AI responses before use. They were identified through discimination of information of risk from other ChatGPT applications along with developer and user testing.,Planned or in-progress,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,"Other, if other please explain in next question.","HMIS is collaborating with, and recieiving approval from, our local DOE on our internal communication plans. Once our internal developement is ready for customer deployment and communications, HMIS will provided their use cases to all users via an AI website where users can view and request use cases. When a use case gets developed, those effected will be communication with about the AI, and provided informational material, training, and feedback loops to provide reasonable and timely notices. Reasonable and Timely notices include Email, Team Channel, and Online. ",No,"The AI does not take into account any demographic information when generating responses, and does not use any demographic information about employees specifically to generate results. Our AI use case should have have any demographic disparity. If a users were to find an issue with a response, they can provide feedback via a feedback mechanism with each response and provide a summary of their feedback. Developers would then address the issue and look to mitigate any issues found. ",Other,"The development of our AI was done with the consultation of our affected community, where affected community is the community of users who would be using the AI, as well as the community of DOE sites that have employed similar AI tools. We have consulted individually and in groups our community and provided early demonstration, feedback mechanisms, direct user testing, post-transaction feedback collection. Areas (Team Channels) where listening and meetings on topics can occur will be provided before release. ",No – it is not operationally practical to offer this.,"There currently is no escalation process fro the AI use case to evaluate the impact on the individual or group. Use of the AI is a productivty use and is voluntary to the users. Users who do not want to use the AI are not required and any issues personally that arise with an individual or group related to the AI can be captured via our feedback, help, and AI Team communication channels and will be reviewed by IMS Leadership. ",Yes, AI for Intelligent Automation,Department of Energy,DOE,NE INL - NE Idaho National Laboratory (NE),Mission-Enabling (internal agency support) ,,None of the above.,"Improve the timeliness and quality of manual work processes through automation where generative AI can make comparisons and decisions using INL procedures and controlled documents, with humans performing final validation and approval.",Completion of forms for human validation and approval.,Initiated,Neither,10/1/2024,,,,Developed in-house.,,No,,,No,,Yes,Yes,Yes,,"At this time, INL plans to use non-CUI data with this solution, including the employee handbook, approved controlled documents, and other material that will assist workers in completing processes and activities. RAG (mini RAG preferred) is the method that will be used for integration with the AI solution.","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case's development.",,,Yes,No – agency does not have access to source code.,,Yes,"CS-SSP-123,Cybersecurity System Security Plan for Azure Platform",6-12 months,Other,This AI will be utilized by applications and not directly by individuals.,No,,Other,Project in initiation phase.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,No,No testing: No testing of the model to simulate performance in an operational environment has been conducted thus far.,Key functional risks include fabricated & inaccurate answers as well as model & output bias. Mitigated operational risks include model exploit.,Yes – by another appropriate agency office that was not directly involved in the system’s development,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,N/A - individuals are not interacting with the AI for this use case,,No,"Using minimal sensitive information, as identified in ATU, built on commercial solution",None of the above,,No – it is not operationally practical to offer this.,This is an internally used use case. Primary feedback mechanisms are direct use testing and customer feedback.,Yes, Hanford Ai Liaison,Department of Energy,DOE,RL - Hanford - Richland Operations Office - Hanford (EM),Mission-Enabling (internal agency support) ,,None of the above.,"The purpose of the Hanford Ai Liaison tool is to provide a prompt to text tool that will allow users to ask questions against the chat and receive generative responses based on model training. The benefits of this AI are that it provides a mitigation for users that were using similar funtionality in external ChatGPT tools that were available to AI users and then blocked. It also provides a safe, secure enviroment for a ChatGPT experience where our prompts will not be used to train models in external system and our prompt and document data does not leave our accedidation boundary.","The system outputs text respones from user prompts using generative, pre-trained, transformer logic large language modeling (GPT LLM)",Operation and Maintenance,Neither,4/23/2024,5/14/2024,10/28/2024,,Developed with contracting resources.,89303320DEM000031,No,,,No,,No,Yes,No,,No agency-owned data is used in this model. Our Data does not train the models.,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups.",N/A,,Yes,No – agency does not have access to source code.,,Yes,Microsoft Azure OpenAI,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,Planned or in-progress.,"Impact evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution and has utilized randomized experiments with a control group or other counterfactual, social systems analysis, or other rigorous research methodologies to evaluate impact and identify potential harm to users as well as broader groups of people.",Risks identified in this AI were incorrect or halucination responses from the AI. Users are trained and informed to validate the AI responses before use. They were identified through discimination of information of risk from other ChatGPT applications along with developer and user testing.,Yes – by another appropriate agency office that was not directly involved in the system’s development,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,"Other, if other please explain in next question.","HMIS is collaborating with, and recieiving approval from, our local DOE on our internal communication plans. HMIS has provided their use cases to all users via an AI website where users can view and request use cases. When a use case gets developed, those effected are communication with about the AI, and provided informational material, training, and feedback loops to provide reasonable and timely notices. Reasonable and Timely notices include Email, Team Channel, and Online.",No,"The AI does not take into account any demographic information when generating responses, and does not use any demographic information about employees specifically to generate results. Our AI use case should have have any demographic disparity. If a users were to find an issue with a response, they can provide feedback via a feedback mechanism with each response and provide a summary of their feedback. Developers would then address the issue and look to mitigate any issues found.",Other,"The development of our AI was done with the consultation of our affected community, where affected community is the community of users who would be using the AI, as well as the community of DOE sites that have employed similar AI tools. We have consulted individually and in groups our community and provided early demonstration, feedback mechanisms, direct user testing, post-transaction feedback collection, and areas (Team Channels) where listening and meetings on topics can occur.",No – it is not operationally practical to offer this.,"There currently is no escalation process fro the AI use case to evaluate the impact on the individual or group. Use of the AI is a productivty use and is voluntary to the users. Users who do not want to use the AI are not required and any issues personally that arise with an individual or group related to the AI can be captured via our feedback, help, and AI Team communication channels and will be reviewed by IMS Leadership.",Yes, Tritium Accountancy for Fusione Energy,Department of Energy,DOE,SRS - SRNL - Savannah River Site - Savannah River National Laboratory (EM),Science & Space,,None of the above.,"Adaptive learning, improved uncertainty on accountancy, and prediction for commercial systems.",Real-time tritium data with reduced uncertaintingy for a continually operating fusion power plant. A framework to be used in full systems.,Initiated,Neither,10/1/2023,10/1/2023,,,Developed in-house.,No-Cost,No,,,Yes,Early Career Award from Office of Science; DOE Guidelines,No,No,No,,Sensor measurements of hydrogen and deuterium. Subject matter experts.,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,No – agency does not have access to source code.,,No,,6-12 months,No,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,No,No testing: No testing of the model to simulate performance in an operational environment has been conducted thus far.,Still working on AI algorithm assessment tests to identify optimal choice,Planned or in-progress,No monitoring protocols have been established: Necessary infrastructure needed to perform monitoring of AI systems in production is not available and/or a plan to monitor models has not yet been established.,No - Some individual decisions or actions require direct human oversight.,,"Other, if other please explain in next question.",Not deployed and is still in development,No,N/A,Other,Subject matter expert discussion and stakeholder discussions during early development,Yes,,Yes, ServiceNow Now Assist,Department of Energy,DOE,NE INL - NE Idaho National Laboratory (NE),Mission-Enabling (internal agency support) ,,None of the above.,"Improve the timeliness and quality of transactional service desk requests through improved incident search, resolution, feedback as well as AI-assisted coding of workflows.","Outputs are consistent with commercial service desk products, including the search, creation and closure of service requests.",Initiated,Neither,10/1/2024,10/1/2024,,,Developed with both contracting and in-house resources.,N/A,No,,,No,,Yes,Yes,Yes,,"This use case will utilize service desk incident, problem and knowledge management sources for training and use.","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case's development.",N/A,,Yes,No – agency does not have access to source code.,,Yes,"CS-CAR-157, Cybersecurity System Security Plan for ServiceNow Platform",6-12 months,Yes,,No,,Other,Project in initiation phase.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,No,No testing: No testing of the model to simulate performance in an operational environment has been conducted thus far.,"Key functional risks include fabricated & inaccurate answers as well as model & output bias. Mitigated operational risks include data privacy, fraud and exploit risks.",Yes – by another appropriate agency office that was not directly involved in the system’s development,"Automated and Regularly Scheduled Updates: Some aspects of the plan to monitor the AI system post-deployment are automated including re-training of models after detecting drift; however, data science teams are still significantly involved in the monitoring and re-deployment process.",No - Some individual decisions or actions require direct human oversight.,,In person ,,No,"Using minimal PII to identify employee (name, id#), commercial solution",None of the above,,No – it is not operationally practical to offer this.,This is an internally used use case. Primary feedback mechanisms are direct use testing and customer feedback.,Yes, Enhancing ClimRR Capabilities to Better Support Electric Utility Applications and Technical Assistance,Department of Energy,DOE,GDO HQ - Grid Deployment Office (GDO),Science & Space,,None of the above.,"The AI, CALLM (Climate Action through Large Language Models), addresses the problem of communicating complex climate projections and scientific literature to a broad audience, particularly electric sector stakeholders. It simplifies this information to help these stakeholders identify climate resilience solutions. Expected benefits include improved communication of climate science, empowering stakeholders to directly address climate change impacts, and accelerating the scalability of ClimRR (presumably an existing Argonne tool) to serve a wider range of users. This ultimately leads to more effective climate resilience planning and potentially cost savings through better informed decision-making.","The AI output of CALLM is information synthesized from complex climate projections and scientific literature, presented in a simplified and accessible format. This output helps users understand potential climate impacts and identify appropriate climate resilience solutions. The information is grounded in vetted data and published research to minimize inaccuracies and hallucinations common in large language models. The output could range from summaries of climate-related risks, to lists of potential adaptation strategies tailored to specific situations, depending on the user's input and the function of the system it's integrated with (like ClimRR).",Initiated,Neither,8/26/2024,10/1/2024,,,Developed in-house.,,No,,,,,No,,No,,"The Climate Action through Large Language Models (CALLM) tool utilizes vetted climate data and published climate resilience literature to train, fine-tune, and evaluate its performance. The specific datasets are not detailed here, but the approach emphasizes the use of established climate science information to ground the model's responses and mitigate inaccuracies.",,,,,,,No,,,,,,,,,,,,,,,,,,,,,,,,,,,, WCD-AI,Department of Energy,DOE,ANL - Argonne National Laboratory (SC43 OIM),Mission-Enabling (internal agency support) ,,None of the above.,Recommend keyword-based search for relevant Lessons Learned published on DOE OPEXShare.,Recommended keywords for search based on user-authored Work Control Document.,Operation and Maintenance,Neither,6/1/2020,7/1/2020,2/1/2021,,Developed in-house.,N/A,No,,,No,,No,No,Yes,,Existing Work Control Document records in database system.,"Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",Other,,Yes,"Yes – agency has access to source code, but it is not public.",N/A,Yes,Argonne National Laboratory Information Technology Infrastructure,Less than 6 months,No,,No,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,No,"Impact evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution and has utilized randomized experiments with a control group or other counterfactual, social systems analysis, or other rigorous research methodologies to evaluate impact and identify potential harm to users as well as broader groups of people.",AI use case provides recommendation of keywords for user to perform a manual search. Negligible risk.,Planned or in-progress,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,"Other, if other please explain in next question.",AI use case provides recommendation of keywords for user to perform a manual search. Negligible risk.,No,Users can provide feedback.,Other,Follow individual direct user feedback.,No – it is not operationally practical to offer this.,The application simply provides suggsted keywords for further research.,Yes, OpenText for Records Management (File share auto-classification),Department of Energy,DOE,EE HQ - EE Headquarters (EE),Mission-Enabling (internal agency support) ,,None of the above.,Reduce time required to classify legacy records accumulated over 20 years.,OpenText will categorize each file located on the network drives.,Acquisition and/or Development,Neither,10/1/2022,1/2/2023,,,Developed with contracting resources.,other direct costs,No,,,No,,Yes,Yes,No,,Trained using existing internal EERE records currated by subject matter experts.,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups.",Other,,No,No – agency does not have access to source code.,,Yes,EE HQ LAN (aka SCORE),Less than 6 months,Other,not applicable,Other,not applicable,Other,not applicable,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, OPQ-AI,Department of Energy,DOE,ANL - Argonne National Laboratory (SC43 OIM),Mission-Enabling (internal agency support) ,,None of the above.,"Significantly reduces person-hours for manual review of incoming people registrations to match with existing database records by recommending most likely matches, if an existing record is identified that matches with the registration details.",Recommendation of matched person record that already exists or that a new person record should be created.,Operation and Maintenance,Neither,12/1/2019,5/1/2023,12/1/2023,,Developed in-house.,N/A,No,,,No,,No,No,Yes,,Data includes read-only access internal person/HR records existing in current database systems.,"Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",Other,,Yes,"Yes – agency has access to source code, but it is not public.",N/A,Yes,Argonne National Laboratory Information Technology Infrastructure,Less than 6 months,No,,No,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,No,"Impact evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution and has utilized randomized experiments with a control group or other counterfactual, social systems analysis, or other rigorous research methodologies to evaluate impact and identify potential harm to users as well as broader groups of people.","Mis-match of person records, requires human intervention to re-assign records or remove incorrect records.",Yes – by an agency AI oversight board not directly involved in the system’s development,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,"Other, if other please explain in next question.",The user does interact directly with OPQ-AI. OPQ-AI is a back office process.,No,None. No demographic disparities are forseen.,Other,Feedback from software development teams for enhancements to response from AI use case.,Yes,,Other,"The AI functionality is solely used for the prevention, detection, and investigation of duplicate registration records." Funding Finder,Department of Energy,DOE,IM-50 - Architecture Engineering Technology and Innovation (IM),Mission-Enabling (internal agency support) ,,None of the above.,The Funding Finder will aggregate FOAs from different DOE sources and enables users to ask questions when identifying opportunities and developing proposals. ,Answers to questions about DOE FOAs.,Acquisition and/or Development,Neither,3/10/2024,10/1/2023,,,Developed with both contracting and in-house resources.,89303019AIM000005,No,,,,,,,No,,"The Funding Finder tool is in development and is utilizing publicly available DOE websites and FOAs. Example websites include ""https://infrastructure-exchange.energy.gov/"", ""https://eere-exchange.energy.gov/"", ""https://arpa-e-foa.energy.gov/"", ""https://48c-exchange.energy.gov/"", ""https://oced-exchange.energy.gov/"", ""https://ie-exchange.energy.gov/"".","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case's development.",N/A,,,,,Yes,OCIO Google Cloud Platform,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, PDF Analyzer,Department of Energy,DOE,IM-50 - Architecture Engineering Technology and Innovation (IM),Mission-Enabling (internal agency support) ,,None of the above.,PDF Analyzer will enable teams across the DOE to upload large PDFs and ask questions and generate content related to those PDFs. ,PDF Analyzer will output the answers to a user's question along with the relevant sections of the PDF that the answer is based on. ,Acquisition and/or Development,Neither,10/1/2024,10/14/2024,,,Developed with both contracting and in-house resources.,89303019AIM000005,No,,,,,,,No,,"For initial evaluation, the development team used publicly available documents like the Feasibility Study - Nyngan Scandium Project PDF (https://scandiummining.com/site/assets/files/5775/feasbility_study-nyngan_scandium_project.pdf) to test the tool.","Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",N/A,,,,,Yes,OCIO Google Cloud Platform,Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Hanford Service Ticket Lookup,Department of Energy,DOE,RL - Hanford - Richland Operations Office - Hanford (EM),Mission-Enabling (internal agency support) ,,Searching for information using AI.,"The purpose of the Hanford Service Ticket Lookup is to provide a single interface with for customers to ask questions and get to service tickets without having to navigate extensive menu's, tool bars, and search functions. Eventually, this will include service tickets from multiple platforms, providing the customer with a single interface to do all things service request related. Additionally, users can ask questions in natural language instead of needing to input specific search criteria. ","The system outputs text respones from user prompts requesting information on grounded data related to Service Ticket Requests ",Acquisition and/or Development,Neither,5/7/2024,6/22/2024,,,Developed with contracting resources.,"89303320DEM000031 ",No,,,No,,No,Yes,Yes,,"Our Data does not train the models. ","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case's development.",N/A,,Yes,No – agency does not have access to source code.,,Yes,Microsoft Azure OpenAI,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,No,No testing: No testing of the model to simulate performance in an operational environment has been conducted thus far.,"Risks identified in this AI were incorrect or halucination responses from the AI. Users are trained and informed to validate the AI responses before use. They were identified through discimination of information of risk from other ChatGPT applications along with developer and user testing. ",Planned or in-progress,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,"Other, if other please explain in next question.","HMIS is collaborating with, and recieiving approval from, our local DOE on our internal communication plans. Once our internal developement is ready for customer deployment and communications, HMIS will provided their use cases to all users via an AI website where users can view and request use cases. When a use case gets developed, those effected will be communication with about the AI, and provided informational material, training, and feedback loops to provide reasonable and timely notices. Reasonable and Timely notices include Email, Team Channel, and Online. ",No,"The AI does not take into account any demographic information when generating responses, and does not use any demographic information about employees specifically to generate results. Our AI use case should have have any demographic disparity. If a users were to find an issue with a response, they can provide feedback via a feedback mechanism with each response and provide a summary of their feedback. Developers would then address the issue and look to mitigate any issues found. ",Other,"The development of our AI was done with the consultation of our affected community, where affected community is the community of users who would be using the AI, as well as the community of DOE sites that have employed similar AI tools. We have consulted individually and in groups our community and provided early demonstration, feedback mechanisms, direct user testing, post-transaction feedback collection. Areas (Team Channels) where listening and meetings on topics can occur will be provided before release. ",No – it is not operationally practical to offer this.,"There currently is no escalation process fro the AI use case to evaluate the impact on the individual or group. Use of the AI is a productivty use and is voluntary to the users. Users who do not want to use the AI are not required and any issues personally that arise with an individual or group related to the AI can be captured via our feedback, help, and AI Team communication channels and will be reviewed by IMS Leadership. ",Yes, INL AI Virtual Assitant (AiVA),Department of Energy,DOE,NE INL - NE Idaho National Laboratory (NE),Mission-Enabling (internal agency support) ,,Improving the quality of written communications using AI tools.,"This chatbot uses commercial chatgpt-like capability to answer questions, provide coaching on processes, summarize and improve communications, and produce code in a variety of formats. INL has been authorized and has planned activiites in 2025 to begin adding internal INL non-CUI data using RAG. Examples include the employee handbook and approved controlled documents.","Outputs are consistent with commercial chatbot products, such as ChatGPT.",Operation and Maintenance,Neither,1/1/2024,1/1/2024,5/13/2024,,Developed in-house.,,No,,,No,,No,Yes,Yes,,"At this time, INL plans to use non-CUI data with this solution, including the employee handbook, approved controlled documents, and other material that will assist workers in completing processes and activities. RAG (mini RAG preferred) is the method that will be used for integration with the AI solution.","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case's development.",N/A,,Yes,No – agency does not have access to source code.,,Yes,"CS-SSP-123,Cybersecurity System Security Plan for Azure Platform",Less than 6 months,Yes,,No,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,No,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.","Key functional risks are similar to use of ChatGPT, including fabricated & inaccurate answers, model & output bias, and potential copyright risks. Mitigated operational risks include data privacy, fraud and exploit risks. Additional considerations include ethics and professional responsibilities, as associated with any new technology/process differentiator.",Yes – by another appropriate agency office that was not directly involved in the system’s development,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,In person ,,No,"Not using PII, models and interface are commercial",None of the above,,No – it is not operationally practical to offer this.,This is an internally used use case. Primary feedback mechanisms are direct use testing and customer feedback.,Yes, Unleashing AI Transformer Models on FPGAs for Accelerating LHC and Particle Physics,Department of Energy,DOE,FNAL - Fermi National Accelerator (SC43 OIM),Science & Space,,None of the above.,"This effort focuses on Transformer models for representation learning on Field Programmable Gate Arrays (FPGA), in order to seamlessly integrate AI capabilities into particle physics experiments, specifically focusing on the CMS level-1 (L1) trigger at the High-Luminosity LHC (HL-LHC) and real-time magnet quench detection. While conventional methods for event identification have limitations, modern AI and machine learning techniques offer superior alternatives.","This AI system has two fold use cases, represnetation learning for LHC Trigger and multi-modal magnet quench detection algorithms.",Acquisition and/or Development,Neither,10/1/2023,10/1/2024,10/1/2027,,Developed in-house.,,No,,,No,,No,No,No,,research datasets from scientific experiments,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case's development.",Other,,Yes,"Yes – agency has access to source code, but it is not public.",N/A,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,N/A,,,,None of the above,,,,, Hanford Procedure Search,Department of Energy,DOE,RL - Hanford - Richland Operations Office - Hanford (EM),Mission-Enabling (internal agency support) ,,Searching for information using AI.,"The purpose of the Hanford Search is to provide greater service in our customers search for relevant proceedures, which is a main look-up for many of our employees. The benefits of the AI are that there is a single interface with multiple uses and the AI can provide better, more relevant search results. Additionally, users can ask questions in natural language instead of needing to input specific search criteria. ","The system outputs text respones from user prompts requesting information on grounded data related to the Hanford Procedure System ",Acquisition and/or Development,Neither,6/24/2024,6/23/2024,,,Developed with contracting resources.,"89303320DEM000031 ",No,,,No,,No,Yes,Yes,,"Our Data does not train the models. ","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case's development.",N/A,,Yes,No – agency does not have access to source code.,,Yes,Microsoft Azure OpenAI,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,No,No testing: No testing of the model to simulate performance in an operational environment has been conducted thus far.,"Risks identified in this AI were incorrect or halucination responses from the AI. Users are trained and informed to validate the AI responses before use. They were identified through discimination of information of risk from other ChatGPT applications along with developer and user testing. ",Planned or in-progress,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,"Other, if other please explain in next question.","HMIS is collaborating with, and recieiving approval from, our local DOE on our internal communication plans. Once our internal developement is ready for customer deployment and communications, HMIS will provided their use cases to all users via an AI website where users can view and request use cases. When a use case gets developed, those effected will be communication with about the AI, and provided informational material, training, and feedback loops to provide reasonable and timely notices. Reasonable and Timely notices include Email, Team Channel, and Online. ",No,"The AI does not take into account any demographic information when generating responses, and does not use any demographic information about employees specifically to generate results. Our AI use case should have have any demographic disparity. If a users were to find an issue with a response, they can provide feedback via a feedback mechanism with each response and provide a summary of their feedback. Developers would then address the issue and look to mitigate any issues found. ",Other,"The development of our AI was done with the consultation of our affected community, where affected community is the community of users who would be using the AI, as well as the community of DOE sites that have employed similar AI tools. We have consulted individually and in groups our community and provided early demonstration, feedback mechanisms, direct user testing, post-transaction feedback collection. Areas (Team Channels) where listening and meetings on topics can occur will be provided before release. ",No – it is not operationally practical to offer this.,"There currently is no escalation process fro the AI use case to evaluate the impact on the individual or group. Use of the AI is a productivty use and is voluntary to the users. Users who do not want to use the AI are not required and any issues personally that arise with an individual or group related to the AI can be captured via our feedback, help, and AI Team communication channels and will be reviewed by IMS Leadership. ",Yes, LLM EV ,Department of Energy,DOE,NREL - National Renewable Energy Laboratory (EE),Mission-Enabling (internal agency support) ,,Collaborating in real-time using AI-assisted tools in word processors.,"Making NREL EV-specific data more accessible for researchers and accelerating research in this field. ","Outputs analysis results from these studies: https://www.sciencedirect.com/science/article/pii/S2666546824000971 ",Acquisition and/or Development,Neither,6/21/2024,7/1/2024,10/1/2024,,Developed in-house.,NA,No,,,No,,No,No,No,,NA,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups.",,,Yes,Yes – source code is publicly available.,https://github.com/NREL/elm/tree/main/examples/ordinance_gpt,Yes,"NREL Stratus AWS Compute Environment, NREL Azure System",Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,Planned or in-progress.,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.","Misrepresentation of local legal policies of EV permitting, in open access datasets. ",Planned or in-progress,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,Online – in the terms or instruction for the service,"Including legal disclaimers with associated outputs. ",No,"Initial spacial validation ",None of the above,NA,Yes,,"No – This AI use case is not subject to the opt-out requirement because the AI functionality is solely used for the prevention, detection, and investigation of fraud or cybersecurity incidents, or the conduct of a criminal investigation.", AskPRIMR,Department of Energy,DOE,NREL - National Renewable Energy Laboratory (EE),Mission-Enabling (internal agency support) ,,Collaborating in real-time using AI-assisted tools in word processors.,"Making data more accessible and user-friendly for the public. ","The systems leverages Retrieval-Augmented Generation to find semantically relevent content which the AI (LLM) summarizes for the end user as a method to describe relevent content within the data catalog. ",Operation and Maintenance,Neither,9/30/2024,10/1/2024,10/30/2024,,Developed in-house.,"NA ",No,,,Yes,"""To demonstrate compliance with the DOE Information Quality Act (IQA) guidelines, the approach with RAG and LLM-based applications ensures the quality, objectivity, utility, and integrity of disseminated information. Quality and Objectivity: The approach uses a locked-down Retrieval-Augmented Generation (RAG) system that minimizes inaccuracies (hallucinations) and excludes memory components, ensuring accurate, clear, and reliable information presentation. This satisfies the IQA's requirement for objective, unbiased dissemination and maintains transparency. Utility: Integrating metadata-driven semantic search in the vector database enhances the utility of DOE's data catalog for public users, meeting the IQA's standard for usefulness. This utility-focused design ensures that data is accessible, relevant, and usable for public insights, supporting transparency and public knowledge. Transparency and Reproducibility: The approach supports data reproducibility by basing responses on catalog metadata and summary information, ensuring consistency and enabling independent reanalysis by qualified users. Transparency is further upheld through documentation of the RAG interactions, aligning with the IQA's standards for open access and transparency in information dissemination. Response to Public Needs: This design incorporates continuous feedback to improve relevance and usability, engaging public and stakeholder needs effectively. It aligns with the IQA's goal of considering public feedback in information quality and ensuring accountability through documented, high-level review processes for data corrections if requested. By adhering to these DOE IQA guidelines, this approach ensures public trust in DOE's open data, demonstrating alignment with the standards of quality, utility, objectivity, and integrity as required under OMB M-19-15."" ",No,No,Yes,,"As this is summarizing data from the PRIMRE data repository the related datasets are described by the catalog which is also used to validate the AI responses: https://openei.org/wiki/PRIMRE ","Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",,,Yes,"Yes – agency has access to source code, but it is not public.",N/A,Yes,"Amazon Web Services, NREL Azure System",Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,Planned or in-progress.,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.","As the use case for the AI has been limited to summarization of existing metadata the key risks were identified as LLM hallucinations or LLM generated text which would cause reputational harm to DOE. Techniques such as red-teaming, adversarial testing, and prompt-injection testing were used to evaluate constraint of hallucinations and LLM. ",Planned or in-progress,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,Online – in the terms or instruction for the service,,No,"Process to detect bias and fairness are in testing and being regulary evaluated by model provideers. There are built-in guardrails in place to prevent abuse ",,,No – it is not operationally practical to offer this.,"As the action of the system is to summarize and communicate existing information returned for a query, the ability of the AI to make impact individuals or groups is significantly limited. ",Other,"Other methods of searching content using common search methodlogies are also available to the catalog users. " Coupa,Department of Energy,DOE,KCNSC - Kansas City National Security Campus (KCFO),Mission-Enabling (internal agency support) ,,Creating visual representations of data sets for reports and presentations using AI.,AI functionality within Coupa drives better search functionailty and drives efficiency of sourcing and procurment processes.,Predictive analysis ,Operation and Maintenance,Neither,6/30/2020,11/30/2020,9/16/2022,,Developed with contracting resources.,Procured under SCMC Agreement # WS200305,No,,,No,,No,No,Yes,,Non production related procurement data is currently being collected and managed by the SCMC. This is the data set that is used by the system.,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups.",,,No,No – agency does not have access to source code.,,Yes,COUPA BSM FOR FEDERAL,Less than 6 months,No,,No,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,No,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.",Not applicable to the risks defined above,Planned or in-progress,"Automated and Regularly Scheduled Updates: Some aspects of the plan to monitor the AI system post-deployment are automated including re-training of models after detecting drift; however, data science teams are still significantly involved in the monitoring and re-deployment process.",No - Some individual decisions or actions require direct human oversight.,,"Other, if other please explain in next question.",Not applicable to the definitions defined above,No,Not applicable to the scenario defined above,Other,Not applicable to the scenario defined above,"No – Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.",There is no AI system outcome that would have an impact on any specific individual or group,Yes, GitHub Copilot with the OpenAI Codex,Department of Energy,DOE,PNNL - Pacific Northwest National Laboratory (SC43 OIM),Science & Space,,None of the above.,"This increases their productivity, allows them to focus more on innovation and research, and accelerates the development of high-quality software solutions for scientific research.","This increases their productivity, allows them to focus more on innovation and research, and accelerates the development of high-quality software solutions for scientific research.",Initiated,Neither,10/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, First Alert DataMinr AI,Department of Energy,DOE,NA-IM (Enterprise) – Office of the Associate Administrator for Information Management and Chief Information Officer (HQ-Enterprise) (NNSA HQ),Emergency Management,,Searching for information using AI.,"We use this AI service to provide spectial or situational awareness, incidents occuring around NNSA or DOE Site. Emergency reporting, monitoring, speeds up emergency operations reporting. The AI is being used in place of a large team that would be required for coding and development of emergency services facilitation. This would be a cost saving software for the Federal government.",Data aggregation and reflection.,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Yurts AI search function,Department of Energy,DOE,SLAC - SLAC National Accelerator Laboratory (SC43 OIM),Mission-Enabling (internal agency support) ,,Searching for information using AI.,"Locating relevant information across multiple data sources at SLAC. The expected benefit SLAC intends to receive from the use of Yurts is improved data accessibility for the userbase at SLAC ","Data summaries and recommendations ",Implementation and Assessment,Neither,6/2/2023,2/27/2024,,,Developed with both contracting and in-house resources.,N/A,No,,,No,,No,No,No,,"The Yurts.ai model that's implemented at SLAC is fed SLAC-internal documentation for training and use ","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups.",Other,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, "Interactive platform to help review and create ""Promoting Inclusive and Equitable Research"" Plans",Department of Energy,DOE,PPPL - Princeton Plasma Physics Laboratory (SC43 OIM),Mission-Enabling (internal agency support) ,,None of the above.,"A trained and informed model that can help to review PIER plans for accuracy, consistency and also to help integrate specific PPPL DEIA goals and initiatives, aligned with input from the user, helping to clearly define the goals of the research plan.",AI output will be revision suggestions to a user submitted PIER plan in order to align with PPPL specific goals and initiatives. It will also help to guide the user to create a more consistent plan with previously submitted/approved plans.,Initiated,Neither,5/1/2024,9/1/2024,,,Developed in-house.,,No,,,No,,No,No,No,,"PPPL specific PIER plan guidelines, previously submitted and approved PIER plans, public DOE guidance and other leadership data to help refine plans that align with laboratory strategic goals.","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case's development.",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,Planned or in-progress.,No testing: No testing of the model to simulate performance in an operational environment has been conducted thus far.,No key risks identified,Planned or in-progress,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,,Email,No,"RAG Model will be significantly grounded in factual data, limiting creativity in responses",,Independent feedback links will be given to receive feedback data outside of the model,Yes,,Yes, Energy Wizard,Department of Energy,DOE,NREL - National Renewable Energy Laboratory (EE),Mission-Enabling (internal agency support) ,,Collaborating in real-time using AI-assisted tools in word processors.,Making NREL data more accessible for researchers and accelerating research. ,The systems leverages Retrieval-Augmented Generation to find semantically relevent content which the AI (LLM) summarizes for the end user as a method to describe relevent content within the selected publications and research profiles. ,Operation and Maintenance,Neither,6/21/2024,7/1/2024,9/23/2024,,Developed in-house.,NA,No,,,Yes,"""To demonstrate compliance with the DOE Information Quality Act (IQA) guidelines, the approach with RAG and LLM-based applications ensures the quality, objectivity, utility, and integrity of disseminated information. Quality and Objectivity: The approach uses a locked-down Retrieval-Augmented Generation (RAG) system that minimizes inaccuracies (hallucinations) and excludes memory components, ensuring accurate, clear, and reliable information presentation. This satisfies the IQA's requirement for objective, unbiased dissemination and maintains transparency. Utility: Integrating metadata-driven semantic search in the vector database enhances the utility of DOE's data catalog for public users, meeting the IQA's standard for usefulness. This utility-focused design ensures that data is accessible, relevant, and usable for public insights, supporting transparency and public knowledge. Transparency and Reproducibility: The approach supports data reproducibility by basing responses on catalog metadata and summary information, ensuring consistency and enabling independent reanalysis by qualified users. Transparency is further upheld through documentation of the RAG interactions, aligning with the IQA's standards for open access and transparency in information dissemination. Response to Public Needs: This design incorporates continuous feedback to improve relevance and usability, engaging public and stakeholder needs effectively. It aligns with the IQA's goal of considering public feedback in information quality and ensuring accountability through documented, high-level review processes for data corrections if requested. By adhering to these DOE IQA guidelines, this approach ensures public trust in DOE's open data, demonstrating alignment with the standards of quality, utility, objectivity, and integrity as required under OMB M-19-15."" ",No,No,Yes,,As this is summarizing data from the OEDI data repository the related datasets are described by the catalog which is also used to validate the AI responses: https://data.openei.org/,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case's development.",,,Yes,No – agency does not have access to source code.,N/A,Yes,"NREL Stratus AWS Compute Environment, NREL Azure System, and NREL Google Cloud Platform",Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,Planned or in-progress.,"Benchmark evaluation: Testing of the AI model has involved the use of benchmarks (either publicly available or internally created) to estimate performance in real-world settings, but has not been tested in an operational environment. ", Intellectual Property and Copyright Concerns. Bias and Fairness,Planned or in-progress,No monitoring protocols have been established: Necessary infrastructure needed to perform monitoring of AI systems in production is not available and/or a plan to monitor models has not yet been established.,No - Some individual decisions or actions require direct human oversight.,,Online – in the terms or instruction for the service,,No,"Process to detect bias and fairness are in testing and being regulary evaluated by model provideers. There are built-in guardrails in place to prevent abuse ",None of the above,,Yes,,Other,Process to detect bias and fairness are in testing and being regulary evaluated by model provideers. There are built-in guardrails in place to prevent abuse LANL AI Portal,Department of Energy,DOE,LANL - Los Alamos National Lab (LAFO),Mission-Enabling (internal agency support) ,,None of the above.,"Democratized access to open-source/open-weights Large Language Models (LLMs) for general purpose office productivity, research of AI models, software development, operational streamlining, code development ","Interactive text chat replies from user prompts, summaries of documents user submitted for Retrieval Augmented Generation (RAG), Replies to API queries from enterprise and scientific applications ",Implementation and Assessment,Neither,11/6/2023,3/21/2024,3/21/2024,,Developed in-house.,,No,,,No,,No,No,No,,"Not trained in-house, using open-source/open-weights models. Reliant on model provider transparency ","Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",Other,,Yes,"Yes – agency has access to source code, but it is not public.",N/A,Yes,"Authorization to Operate Amazon Web Service Federal Risk and Authorization Management Program Infrastructure as a Service Cloud Service Provider, Amazon Web Service Information System Security Plan, SD-006-CP-002/L2",Less than 6 months,No,,Yes,,Other,"Security change request approval for introduction of the software tools used to build the portal took >3 months to clear local field office. Compute provisioning by the LANL Cloud Services team was fast ",Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,,,,,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,,,No,,,,Yes,,Other,"The AI Portal is an opt-in service if they wish to interact with a large language model, users of the portal are under no obligation to go to the website " Position Description Tool,Department of Energy,DOE,IM-50 - Architecture Engineering Technology and Innovation (IM),Mission-Enabling (internal agency support) ,,None of the above.,Currently requirements are still be collected for this use case. The end goal of this use case is to streamline the creation of position descriptions used as a critical component for hiring federal employees. ,Position descriptions for federal roles.,Initiated,Neither,11/5/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, ChatGPT Enterprise,Department of Energy,DOE,LANL - Los Alamos National Lab (LAFO),Mission-Enabling (internal agency support) ,,None of the above.,"General business productivity, research of AI models ","Interactive text chat replies from user prompts, summaries of open/public/unrestricted documents user submitted for Retrieval Augmented Generation (RAG) through CustomGPTs ",Operation and Maintenance,Neither,11/6/2023,4/16/2024,4/18/2024,,Developed in-house.,,No,,,No,,No,No,No,,"Not trained on any agency or LANL data ","Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",Other,,No,No – agency does not have access to source code.,N/A,No,,Less than 6 months,No,,Other,"N/A (SAAS platform delivery) ",Other,"N/A (SAAS platform delivery) ",Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,,,,,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,,,No,,,,Yes,,Other,"ChatGPT Enterprise is an opt-in service where users need to procure their licenses in order to choose interaction with OpenAI tools " Argo,Department of Energy,DOE,ANL - Argonne National Laboratory (SC43 OIM),Mission-Enabling (internal agency support) ,,None of the above.,"Enables anyone with the Argonne community to leverage text-based generative AI with their Argonne-specific information and data, including sensitive research or operational data up to and including CUI.",Large language model responses (prediction-based) following user prompting.,Operation and Maintenance,Neither,4/1/2023,5/1/2023,12/1/2023,,Developed in-house.,N/A,No,,,No,,No,No,Yes,,"N/A – we are using existing pre-trained large language models, no training required.","Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",Other,,Yes,"Yes – agency has access to source code, but it is not public.",N/A,Yes,Argonne National Laboratory Information Technology Infrastructure,Less than 6 months,No,,No,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,None of the above,,,,, AI Chat Bot for Facility Sustainability Practices,Department of Energy,DOE,PPPL - Princeton Plasma Physics Laboratory (SC43 OIM),Mission-Enabling (internal agency support) ,,None of the above.,"Interactive RAG chat bot model trained on facility recycling, composting and trashing guidelines to inform users how to handle niche cases for sustainably getting rid of unwanted items. This should reduce confusion when throwing items out and also increase the amount of properly recycled items at PPPL.","AI output will be recommendations and instructions based on training data from facility data on recycling, trash and composting",Initiated,Neither,5/1/2024,9/1/2024,,,Developed in-house.,,No,,,No,,No,No,No,,"Facility documentation on proper recycling, trash and composting guidelines. Location data for areas where specific items can be thrown away. Dynamic training on publicly maintained websites to interpret updated guidelines within PPPL and in the complex, as well as data on upcoming events.","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case's development.",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,Planned or in-progress.,No testing: No testing of the model to simulate performance in an operational environment has been conducted thus far.,No key risks identified,Planned or in-progress,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,,Email,No,"RAG Model will be significantly grounded in factual data, limiting creativity in responses",,Independent feedback links will be given to receive feedback data outside of the model,Yes,,Yes, AskGDR,Department of Energy,DOE,NREL - National Renewable Energy Laboratory (EE),Mission-Enabling (internal agency support) ,,Collaborating in real-time using AI-assisted tools in word processors.,"Making data more accessible and user-friendly for the public. ","The systems leverages Retrieval-Augmented Generation to find semantically relevent content which the AI (LLM) summarizes for the end user as a method to describe relevent content within the data catalog. ",Operation and Maintenance,Neither,9/30/2024,10/1/2024,10/30/2024,,Developed in-house.,"NA ",No,,,Yes,"""To demonstrate compliance with the DOE Information Quality Act (IQA) guidelines, the approach with RAG and LLM-based applications ensures the quality, objectivity, utility, and integrity of disseminated information. Quality and Objectivity: The approach uses a locked-down Retrieval-Augmented Generation (RAG) system that minimizes inaccuracies (hallucinations) and excludes memory components, ensuring accurate, clear, and reliable information presentation. This satisfies the IQA's requirement for objective, unbiased dissemination and maintains transparency. Utility: Integrating metadata-driven semantic search in the vector database enhances the utility of DOE's data catalog for public users, meeting the IQA's standard for usefulness. This utility-focused design ensures that data is accessible, relevant, and usable for public insights, supporting transparency and public knowledge. Transparency and Reproducibility: The approach supports data reproducibility by basing responses on catalog metadata and summary information, ensuring consistency and enabling independent reanalysis by qualified users. Transparency is further upheld through documentation of the RAG interactions, aligning with the IQA's standards for open access and transparency in information dissemination. Response to Public Needs: This design incorporates continuous feedback to improve relevance and usability, engaging public and stakeholder needs effectively. It aligns with the IQA's goal of considering public feedback in information quality and ensuring accountability through documented, high-level review processes for data corrections if requested. By adhering to these DOE IQA guidelines, this approach ensures public trust in DOE's open data, demonstrating alignment with the standards of quality, utility, objectivity, and integrity as required under OMB M-19-15."" ",No,No,Yes,,"As this is summarizing data from the GDR data repository the related datasets are described by the catalog which is also used to validate the AI responses: https://gdr.openei.org/ ","Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",,,Yes,"Yes – agency has access to source code, but it is not public.",N/A,Yes,NREL Stratus AWS Compute Environment,Less than 6 months,Yes,,No,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,Planned or in-progress.,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.","As the use case for the AI has been limited to summarization of existing metadata the key risks were identified as LLM hallucinations or LLM generated text which would cause reputational harm to DOE. Techniques such as red-teaming, adversarial testing, and prompt-injection testing were used to evaluate constraint of hallucinations and LLM. ",Planned or in-progress,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,Online – in the terms or instruction for the service,,No,"Process to detect bias and fairness are in testing and being regulary evaluated by model provideers. There are built-in guardrails in place to prevent abuse ",,,No – it is not operationally practical to offer this.,"As the action of the system is to summarize and communicate existing information returned for a query, the ability of the AI to make impact individuals or groups is significantly limited. ",Other,"Other methods of searching content using common search methodlogies are also available to the catalog users. " Topic Modeling for Energy.gov ,Department of Energy,DOE,PA HQ - Office of Public Affairs (PA),Mission-Enabling (internal agency support) ,,None of the above.,"Minimize time spent through manual effort of reading and tagging 100,000 Energy.gov webpages.",A list of five tags that best categorize an Energy.gov webpage. ,Acquisition and/or Development,Neither,5/15/2024,7/16/2024,,,Developed with contracting resources.,89303019AIM000005,No,,,,,,,No,,Web content from Energy.gov is being used for this use case. ,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case's development.",N/A,,,,,Yes,OCIO Google Cloud Platform,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, In-Situ Monitoring of Additively Manufactured Parts,Department of Energy,DOE,SRS - SRNL - Savannah River Site - Savannah River National Laboratory (EM),Science & Space,,None of the above.,Identify print defects during the printing process.,Layerwise annotated images of defects in print bed that can be registered to post print analysis or trigger a fixing process or a full stop to save on material.,Implementation and Assessment,Neither,10/1/2022,10/1/2022,10/1/2022,,Developed with contracting resources.,Government use license,No,,,No,,No,No,No,,Print data,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups.",Other,,Yes,"Yes – agency has access to source code, but it is not public.",N/A,Yes,ORNL Software,Less than 6 months,No,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,No,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.",Annotation,Planned or in-progress,"Automated and Regularly Scheduled Updates: Some aspects of the plan to monitor the AI system post-deployment are automated including re-training of models after detecting drift; however, data science teams are still significantly involved in the monitoring and re-deployment process.",No - Some individual decisions or actions require direct human oversight.,,"Email , Other, if other please explain in next question.","Email and regularly scheduled meetings. ",No,N/A,Other,N/A ORNL Software,Yes,,Yes, Boston Dynamics Spot Robotics,Department of Energy,DOE,Y-12 - Consolidated Nuclear Security Y-12 (NPO),Mission-Enabling (internal agency support) ,,None of the above.,Automation of the robots movements,"Development and testing of robotic use for security and emergency responses, helping to decide on the ""best"" path for the robot to move",Acquisition and/or Development,Neither,2/10/2021,10/1/2021,,,Developed with both contracting and in-house resources.,PO 4300174099 (renewed annually),No,,,No,,No,No,No,,None,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",Other,,Yes,No – agency does not have access to source code.,,No,,Less than 6 months,Other,No IT Infrastructure needed,Other,Not Required,Other,Not Required,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,No,"Benchmark evaluation: Testing of the AI model has involved the use of benchmarks (either publicly available or internally created) to estimate performance in real-world settings, but has not been tested in an operational environment. ",Public acceptance of robotics integration,Yes – by another appropriate agency office that was not directly involved in the system’s development,"Automated and Regularly Scheduled Updates: Some aspects of the plan to monitor the AI system post-deployment are automated including re-training of models after detecting drift; however, data science teams are still significantly involved in the monitoring and re-deployment process.",No - Some individual decisions or actions require direct human oversight.,,N/A - individuals are not interacting with the AI for this use case,,No,,"Direct user testing, Post transaction customer feedback collections",,Yes,,Yes, Elastic Stack Technology (ELK),Department of Energy,DOE,Y-12 - Consolidated Nuclear Security Y-12 (NPO),Mission-Enabling (internal agency support) ,,Searching for information using AI.,Enable intelligent media cataloging to assist with data discovery,"Intelligent collection content searching using ElasticSearch, Logstash, and Kibana",Acquisition and/or Development,Neither,10/1/2022,12/1/2022,,,Developed with both contracting and in-house resources.,PO 4300182430 (renewed annually),No,,,No,,No,No,No,,Knowledge Preservation Management (KPM) Media,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case's development.",,,Yes,No – agency does not have access to source code.,,Yes,Elastic Stack Software,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,No,"Benchmark evaluation: Testing of the AI model has involved the use of benchmarks (either publicly available or internally created) to estimate performance in real-world settings, but has not been tested in an operational environment. ",Not identifying the right media for target searches,Planned or in-progress,"Automated and Regularly Scheduled Updates: Some aspects of the plan to monitor the AI system post-deployment are automated including re-training of models after detecting drift; however, data science teams are still significantly involved in the monitoring and re-deployment process.",No - Some individual decisions or actions require direct human oversight.,,Online – in the terms or instruction for the service,,Yes,Manual review by Human in the Loop resolves discrepancies,"Direct user testing, Post transaction customer feedback collections",,Yes,,Yes, Raytheon Multimedia Monitoring System (M3S),Department of Energy,DOE,Y-12 - Consolidated Nuclear Security Y-12 (NPO),Mission-Enabling (internal agency support) ,,Transcribing and summarizing a recorded meeting or interview using AI.,Enable off-cloud transcription of pre-recorded video media to improve data discoverability,Off-cloud voice to text transcription technology for pre-recorded videos,Acquisition and/or Development,Neither,10/1/2017,12/1/2017,,,Developed with both contracting and in-house resources.,PO 4300183978 (renewed annually),No,,,No,,No,No,No,,Knowledge Preservation Management (KPM) Media,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case's development.",,,Yes,No – agency does not have access to source code.,,Yes,The Multimedia Monitoring System,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,No,"Benchmark evaluation: Testing of the AI model has involved the use of benchmarks (either publicly available or internally created) to estimate performance in real-world settings, but has not been tested in an operational environment. ","Incomplete or incorrect transcription of videos, negatively impacting the discoverability of this media",Planned or in-progress,"Automated and Regularly Scheduled Updates: Some aspects of the plan to monitor the AI system post-deployment are automated including re-training of models after detecting drift; however, data science teams are still significantly involved in the monitoring and re-deployment process.",No - Some individual decisions or actions require direct human oversight.,,Online – in the terms or instruction for the service,,Yes,Manual review by Human in the Loop resolves discrepancies,"Direct user testing, Post transaction customer feedback collections",,Yes,,Yes, Cognitive Prescreen Tool (CPT),Department of Energy,DOE,Y-12 - Consolidated Nuclear Security Y-12 (NPO),Mission-Enabling (internal agency support) ,,None of the above.,"Serve as a recommender to Derivative Classifier to assist with document review to improve process accuracy and efficiency, in that order.",Sensitive information detection bound to DOE classification guidance to help reduce IOSC and prevent information loss,Acquisition and/or Development,Neither,10/1/2016,10/1/2019,,,Developed in-house.,,No,,,No,,No,,No,,Classification Guides from the CNS Classification Office,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case's development.",Other,,Yes,No – agency does not have access to source code.,,Yes,Automated Classification System,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,No,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.",IOSC potential as information detection models are tuned,Yes – by another appropriate agency office that was not directly involved in the system’s development,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,Online – in the terms or instruction for the service,,Yes,Manual review by Human in the Loop resolves discrepancies,"Direct user testing, Post transaction customer feedback collections",,Yes,,Yes, Machine Learning for Linac Improved Performance,Department of Energy,DOE,FNAL - Fermi National Accelerator (SC43 OIM),Science & Space,,None of the above.,"Daily fluctuations in the Ion Source conditions as well af the effect of environmental changes to RF systems and cavities affect Linac beam. Results include increased beam loss resulting in increased beamline component irradiation, decreased beam intensity to downstream machines affecting Accelerator Complex deliverables, drifts in Linac beam energy directly affecting Booster losses. These drifts are not easily predictable since we do not have environmental control on the RF gallery, not enough instrumentaion in the Ion Source or Linac proper. To counter these effects, we are developing AI-based optimization and modeling, including Bayesian Optimization and surrogate model-based optimization, with the ultimate goal of (near) real-time RF compensation.",Outputs are proposed changes to RF system parameters (cavity phase settings and/or field gradients) to counter the effect of daily drift and to stabilize the output energy.,Acquisition and/or Development,Neither,10/1/2020,10/1/2020,10/1/2024,,Developed in-house.,,No,,,No,,No,No,No,,Accelerator operations machine data as well as accelerator simulation,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case's development.",Other,,Yes,"Yes – agency has access to source code, but it is not public.",N/A,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,N/A,,,,None of the above,,,,, Next-Generation Beam Cooling and Control with Optical Stochastic Cooling,Department of Energy,DOE,FNAL - Fermi National Accelerator (SC43 OIM),Science & Space,,None of the above.,This effort focuses on enhanced real-time control of the structure of circulating particle beams. The additional performance and capabilities provided may enable substantially greater operational flexibility and science reach at current and future DOE accelerator facilities.,The AI system will continuously infer the state of a circulating beam distribution and then use this inference in the execution of an RL-based control policy. The primary means of control is an advanced optical stochastic cooling system.,Acquisition and/or Development,Neither,10/1/2021,10/1/2021,10/1/2024,,Developed in-house.,,No,,,No,,No,No,No,,Large-scale simulation data is being used to train the diagnostic and control systems. Online training with experimental data may also be leveraged once the system is operational.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case's development.",Other,,Yes,"Yes – agency has access to source code, but it is not public.",N/A,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,N/A,,,,None of the above,,,,, high level synthesis for machine learning (previously hls4ml),Department of Energy,DOE,FNAL - Fermi National Accelerator (SC43 OIM),Science & Space,,None of the above.,hls4ml is used to implement specialized AI algorithms in embedded hardware. This is valuable across a wide range of scientific applications for enabling real-time processing capabilitles. This can accelerate sientific discovery and time to science thus enabling large cost savings and DOE scientific prestige.,It can be an AI algorithm from prediction to data compression to control (decision making).,Acquisition and/or Development,Neither,10/1/2021,6/1/2021,10/1/2024,,Developed in-house.,,No,,,No,,No,No,No,,research datasets from scientific experiments,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups.",Other,,Yes,Yes – source code is publicly available.,https://github.com/fastmachinelearning/hls4ml,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,N/A,,,,None of the above,,,,, Streamining intelligent detectors for sPHENIX/EIC,Department of Energy,DOE,FNAL - Fermi National Accelerator (SC43 OIM),Science & Space,,None of the above.,AI tools are developed for embedded inference in real-time processing systems for scientific experiments such as sPHENIX and upcoming EIC. This can accelerate sientific discovery and time to science thus enabling large cost savings and DOE scientific prestige.,It can be an AI algorithm from prediction to data compression to control (decision making).,Acquisition and/or Development,Neither,10/1/2021,10/1/2021,10/1/2024,,Developed in-house.,,No,,,No,,No,No,No,,research datasets from scientific experiments,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case's development.",Other,,Yes,"Yes – agency has access to source code, but it is not public.",N/A,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,N/A,,,,None of the above,,,,, In-pixel AI for future tracking detectors,Department of Energy,DOE,FNAL - Fermi National Accelerator (SC43 OIM),Science & Space,,None of the above.,AI algorithms are implemented in on-detector electronics in order to reduce data size and enable processing at high rates.,"A recommendation of whether to save data based on AI classifier. Or, a fast inference of track parameters to be used for fast selection",Acquisition and/or Development,Neither,10/1/2022,10/1/2022,10/1/2025,,Developed in-house.,,No,,,No,,No,No,No,,Accelerator operations machine data,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case's development.",Other,,Yes,Yes – source code is publicly available.,https://github.com/smart-pix/smart-pixels-ml,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,N/A,,,,None of the above,,,,, SONIC: AI acceleration as a service,Department of Energy,DOE,FNAL - Fermi National Accelerator (SC43 OIM),Science & Space,,None of the above.,SONIC is used to accelerate AI workloads on coprocessors in scientific experiments. This can accelerate sientific discovery and time to science thus enabling large cost savings and DOE scientific prestige.,It can be an AI algorithm from prediction to data compression to control (decision making).,Acquisition and/or Development,Neither,10/1/2020,10/1/2020,10/1/2024,,Developed in-house.,,No,,,No,,No,No,No,,research datasets from scientific experiments,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case's development.",Other,,Yes,Yes – source code is publicly available.,https://github.com/fastmachinelearning/SonicCMS,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,N/A,,,,None of the above,,,,, READS: Real-time Edge AI for Distributed Systems,Department of Energy,DOE,FNAL - Fermi National Accelerator (SC43 OIM),Science & Space,,None of the above.,"READS has two sub-projects. The first project created the means to stream live Main Injector and Recycler accelerator beam loss monitor data. This data is then fed to an AI model deployed on an FPGA so that it can infer, in realtime, the origin of beam loss, either Main Injector or Recycler, for each beam loss monitor in the tunnel enclosure. The second project aimed to improve upon traditional resonant beam extraction regulation techiniques using AI for use in the Fermilab Delivery Ring and Mu2e.",The ML outputs of the system are inferences as to the origin of beam loss in the Main Injector acclerator enclosure and also suggested regulation ramps to best improve the Spill Duty Factor in the Delivery Ring for Mu2e,Acquisition and/or Development,Neither,10/1/2019,6/1/2019,10/1/2024,,Developed in-house.,,No,,,No,,No,No,No,,research datasets from scientific experiments,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case's development.",Other,,Yes,Yes – source code is publicly available.,https://github.com/fermilab-accelerator-ai/READS,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,N/A,,,,None of the above,,,,, Extreme data reduction for the edge,Department of Energy,DOE,FNAL - Fermi National Accelerator (SC43 OIM),Science & Space,,None of the above.,AI tools are developed for embedded inference in real-time processing systems for scientific experiments. This can accelerate sientific discovery and time to science thus enabling large cost savings and DOE scientific prestige.,It can be an AI algorithm from prediction to data compression to control (decision making).,Acquisition and/or Development,Neither,10/1/2021,6/1/2021,10/1/2024,,Developed in-house.,,No,,,No,,No,No,No,,research datasets from scientific experiments,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case's development.",Other,,Yes,"Yes – agency has access to source code, but it is not public.",N/A,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,N/A,,,,None of the above,,,,, Machine Learning for Accelerator Operations Using Big Data Analytics / L-CAPE,Department of Energy,DOE,FNAL - Fermi National Accelerator (SC43 OIM),Science & Space,,None of the above.,"ML models are deployed for the FNAL's Linac.to detect, label andact upon faults. The usage of ML will jimporve our fault labelingand detection. This will allow for improved operatioal efficeincy, fault statistics, and preventitive maintenance. To my knowledge this is the first global accelerator operations ML system.",The ML outputs to a dashboard withfault labels and downtime predictiojns.The model will also try and predict dwwontime and possible actions.,Acquisition and/or Development,Neither,10/1/2020,1/1/2020,10/1/2024,,Developed in-house.,,No,,,No,,No,No,No,,my own simulated data; research datasets from scientific experiments,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case's development.",Other,,Yes,"Yes – agency has access to source code, but it is not public.",N/A,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,No,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.",None / under evaluation,Planned or in-progress,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,N/A,,No,N/A,None of the above,,Yes,,Yes, Soil Moisture Modeling,Department of Energy,DOE,LM HQ - Office of Legacy Management (LM),Science & Space,,Creating visual representations of data sets for reports and presentations using AI.,Predicts soil moisture contents without adding more monitoring wells.,Prediction,Implementation and Assessment,Neither,,10/3/2022,3/1/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, DOIChatGPT AI Chatbot,Department of the Interior,DOI,OCIO,Mission-Enabling (internal agency support),,Searching for information using AI.,"Create for DOI users a secure space to chat with a Large Language Model without leaving our security boundary. This is a customized version of Microsofts Govchat repo deployed as an azure app service on the internal network with entra authentication, using DOIChatGPT APIM as the back end AI service provider.",See description ,Implementation and Assessment,Neither,,,,,,,,,,,,,,Yes,,,,,,,,,,,,,,,,,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, DOIChatGPT API Management Instance,Department of the Interior,DOI,OCIO,Mission-Enabling (internal agency support),,None of the above.,"An Azure OpenAI instance secured behind an Azure API Management instance hosted within our internal network. This will create a backbone of secure access to Azure OpenAI permitted by APIM subscription keys that also supports cost tracking per project key. This is just a service layer to enable other project activities.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter ,Implementation and Assessment,Neither,,,7/9/2024,,Developed in-house.,,No,,,No,,No,,Yes,,N/A,,,,Yes,"Yes – agency has access to source code, but it is not public.",,,,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, DOIChatGPT API Management Development Environment,Department of the Interior,DOI,OCIO,Mission-Enabling (internal agency support),,None of the above., This will be a public facing version of the DOIChatGPT APIM and Azure OpenAI that is used for training and proof of concept validation (with sanitized data) prior to migrating to production DOIChatGPT APIM.,Outputs were not documented and DOI will update the use cases once additional data is collected from submitter ,Implementation and Assessment,Neither,,,8/21/2024,,Developed in-house.,,No,,,No,,No,,Yes,,N/A,,,,Yes,"Yes – agency has access to source code, but it is not public.",,,,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Cost Estimation Using EDL Database Data,Department of the Interior,DOI,OS - Office of the Secretary of the Interior,Mission-Enabling (internal agency support),,None of the above.," This use case aims to leverage AI/ML for estimating environmental remediation costs based on historical data within the EDL (Environmental Data Liability) database. The model would analyze archived and current records to provide estimates that aid budget planning and contractor evaluations. Problem Being Solved: To enhance budget estimation accuracy for environmental remediation activities by comparing new sites with archived data of similar sites to estimate total cleanup costs. Goal: The goal is to improve cost estimation accuracy, enhance contractor proposal evaluation, and aid in long-term financial planning. ",Intended Use: The AI/ML tool would provide estimates to support the decision-making process for budget planning and contractor proposal assessments. This would act as a supplementary tool and not the primary decision source.,Initiated,Neither,,,,,,,No,,,No,,,,Yes,,"Data Involved: Source: EDL Database; Types: Site type, phase of remediation, complexity, size, contaminants of concern, and cost information. Data characteristics: Structured data fields including categorical (e.g., site type, phase, and contaminants of concern) and numerical (e.g., costs, size estimates). Updates: periodic updates to reflect changes in cost due to inflation or other factors. Annual update for inflation.",,,,,,,,,,,,,,,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Adobe Firefly Generative Images,Department of the Interior,DOI,NPS,Mission-Enabling (internal agency support),,None of the above., Adobe's Firefly generative image AI model was used within Photoshop to create artistic sketches of different nature scenes. These sketches were used in the StoryMap created by San Juan Island National Historical Park as artistic and narrative elements. The images were generated in Photoshop and then manually edited to ensure accuracy. Using this AI model created results that were quicker and cheaper compared to traditional artistic methods. The StoryMap with the sketches can be viewed here: https://arcg.is/1uCy0e,Imagery,Operation and Maintenance,Neither,,,,,Data not reported by submitter and will be updated once additional information is collected,Data not reported by submitter and will be updated once additional information is collected,No,,,Yes,,,,Yes,,Photographs taken by NPS staff were used to compare the accuracy of images output by the AI model,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,"Yes – agency has access to source code, but it is not public.",,,,Less than 6 months,Yes,,Yes,,Yes,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, RAG knowledge base of agent procedures,Department of the Interior,DOI,OCIO,Government Services (includes Benefits and Service Delivery),,Searching for information using AI.,"Custom AI Solutions for Internal and Customer Support Description: We are addressing the challenge of providing quick, accurate, and consistent responses to both internal customer service agents and external customers. Currently, agents must manually search through extensive documentation, which can lead to delays in service response times, inconsistencies in information shared, and a potential for human error. Additionally, external customers often need direct access to information on our website but must wait for human support, which impacts customer satisfaction. By implementing a customized AI chatbot, we aim to streamline access to knowledge for agents, reducing their workload and improving efficiency. Additionally, a customer-facing chatbot would offer external customers instant answers to common questions, further enhancing customer experience and reducing the volume of support inquiries handled by human agents.","Streamline access to knowledge for agents, reducing their workload and improving efficiency. Additionally, a customer-facing chatbot would offer external customers instant answers to common questions, further enhancing customer experience and reducing the volume of support inquiries handled by human agents.",Implementation and Assessment,Neither,,,,,,,,,,,,,,Yes,,,,,,,,,,,,,,,,,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Use of AI to Enhance Flash Flood Forecast Tool,Department of the Interior,DOI,NPS,Mission-Enabling (internal agency support),,None of the above.," University of Illinois created a model to predict rainfall on a watershed scale in Great Smoky Mountains National Park. They are now working with IBM to create a system that will use this model to provide forecasts of flooding events with a goal of a 24+ hour lead time. The developers are using AI to improve on the Quantitative Precipitation Forecast of the National Weather Service to thus improve on the accuracy of their tool. The NPS is cooperating as an end-user of the flood forecast tool, but not as a developer or direct user of the AI.",,Implementation and Assessment,Neither,,,,,Data not reported by submitter and will be updated once additional information is collected,Data not reported by submitter and will be updated once additional information is collected,No,,,,,,,Yes,,Data not reported by submitter and will be updated once additional information is collected,Data not reported by submitter and will be updated once additional information is collected,,,Yes,"Yes – agency has access to source code, but it is not public.",,,,Less than 6 months,Yes,,Yes,,Yes,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Liable Party Research,Department of the Interior,DOI,ONRR,Mission-Enabling (internal agency support),,Summarizing the key points of a lengthy report using AI.,Supervised machine learning model for scraping leasing documents for current liable party ownership.,Efficient liable party ownership.,Implementation and Assessment,Neither,,,,,,,,,,,,,,Yes,,,,,,,,,,,,,,,,,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Data Extraction Using MS Power Automate AI Functionality,Department of the Interior,DOI,ONRR,Mission-Enabling,,None of the above.,Train pre-built Machine Learning model to extract data from different document formats and convert into a consistent structured format to improve business operations.,Improved business documentation. ,Acquisition and/or Development,Neither,,,,,Developed in-house.,,No,,,No,,Yes,No,Yes,,Document copies received from Bureau of Indian Affairs (BIA),,,,No,No – agency does not have access to source code.,,,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, CESU project to detect of bird nests using deep learning to support annual colonial bird monitoring,Department of the Interior,DOI,NPS,Mission-Enabling (internal agency support),,None of the above.,"This work is already being accomplished via a cooperative agreement with a university (Florida International University) The National Park Service (NPS), has been monitoring colonial nesting birds monthly via low level aerial photography from a helicopter platform in Biscayne National Park since 2010. These photographs are processed to determine the number of active nests for specific focal species: Double-crested Cormorants, Great Egrets, Great White Herons, Great Blue Herons, White Ibises, and Roseate Spoonbills. NPS now has 13 years of data which is composed of 64,350 uniquely identified nests and 57,793 birds representing. This monitoring is time consuming and needs to be accomplished consistently and accurately. Currently, the aerial photos are processed by individuals. Which takes substantial time and effort. We propose to explore the creation of automatic script that would allow the active nest to be identified using software. The goal of this project is to explore software programs and create automatic script that can identify potential bird nests on the photograph and to highlight this area on the photograph. By developing a robust object detection model, researchers can monitor bird colonies and their populations more efficiently, e.g. by consistently identifying nests with software which will allow the tracking of nesting patterns with less person effort needed to accomplish this task. It will also help to control observer detection variation over time.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,,,Data not reported by submitter and will be updated once additional information is collected,Data not reported by submitter and will be updated once additional information is collected,No,,,,,,,Yes,,Data not reported by submitter and will be updated once additional information is collected,Data not reported by submitter and will be updated once additional information is collected,,,Yes,"Yes – agency has access to source code, but it is not public.",,,,Less than 6 months,Yes,,Yes,,Yes,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Image Generation and Audio Video Editing,Department of the Interior,DOI,ONRR,Mission-Enabling,,Creating visual representations of data sets for reports and presentations using AI.,"To efficiently create and edit images, audio, and video.",see description ,Operation and Maintenance,Neither,,,,,,,,,,,,,,Yes,,,,,,,,,,,,,,,,,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Use of AI for developing bioacoustic and remote camera imagery wildlife species classifiers for noni,Department of the Interior,DOI,NPS,Mission-Enabling (internal agency support),,None of the above.,"University of Illinois created a model to predict rainfall on a watershed scale in Great Smoky Mountains National Park. They are now working with IBM to create a system that will use this model to provide forecasts of flooding events with a goal of a 24+ hour lead time. The developers are using AI to improve on the Quantitative Precipitation Forecast of the National Weather Service to thus improve on the accuracy of their tool. The NPS is cooperating as an end-user of the flood forecast tool, but not as a developer or direct user of the AI.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,,,Data not reported by submitter and will be updated once additional information is collected,Data not reported by submitter and will be updated once additional information is collected,No,,,,,,,Yes,,Data not reported by submitter and will be updated once additional information is collected,Data not reported by submitter and will be updated once additional information is collected,,,Yes,"Yes – agency has access to source code, but it is not public.",,,,Less than 6 months,Yes,,Yes,,Yes,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Machine Learning Model Optimization,Department of the Interior,DOI,ONRR,Mission-Enabling,,None of the above.," Alteryx machine learning model optimization and fitting helps users select which model is optimal to fit to an underlying dataset. It also can help refine the model, recommending variable removal if autocorrelation issues are detected in the dataset.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter ,Implementation and Assessment,Neither,,,4/13/2023,,Developed in-house.,,No,,,No,,No,,Yes,,"Internal data, subscription sourced data, and publicly available data.",,,,No,No – agency does not have access to source code.,,,,Less than 6 months,Yes,,No,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, NPS Acquisition Management Workload Software,Department of the Interior,DOI,NPS,Government Services (includes Benefits and Service Delivery),,Summarizing the key points of a lengthy report using AI.,Provide assistance for employees working on procurment packages by drafting documents and summarizing content. ,,Implementation and Assessment,Neither,,,,,,,,,,,,,,Yes,,,,,,,,,,,,,,,,,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Caseguard Studio,Department of the Interior,DOI,NPS,Mission-Enabling,,Creating visual representations of data sets for reports and presentations using AI.," Our office uses CaseGuard Studio to streamline and enhance the accuracy of video and audio redactions for law enforcement and public records. This tool addresses the challenge of protecting sensitive information while ensuring compliance with privacy laws and FOIA requirements, improving efficiency and reducing manual workload.",,Implementation and Assessment,Neither,,,,,,,,,,,,,,Yes,,,,,,,,,,,,,,,,,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Azure based Internal Controls Testing App for Award Descriptions,Department of the Interior,DOI,OS - Office of the Secretary of the Interior,Mission-Enabling,,Summarizing the key points of a lengthy report using AI.,PGM will work with Microsoft to develop an application that will live the the secure Azure environment and allow bureaus to conduct their own testing of project descriptions as a learning tool. The application will be locked to the same prompts for review of project descriptions but will allow bureau financial assistance policy personnel to run their own tests to compare to FY 2023 results and look for improvements.,see description ,Acquisition and/or Development,Neither,,,,,,,,,,,,,,Yes,,,,,,,,,,,,,,,,,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Internal Controls Testing - SAM.gov certifications,Department of the Interior,DOI,OS - Office of the Secretary of the Interior,Mission-Enabling,,None of the above.," Federal financial assistance regulation and DOI PGM policy requires awarding officers check SAM.gov, a Federal public website, to ensure that the entity funds are to be award to is eligible to receive Federal funds and that the POCs on the awards are not debarred. The use case involves using a large language AI model to assess and score SAM.gov pdf documents against the date the award was made.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter ,Acquisition and/or Development,Neither,,,,,Developed in-house.,,No,,,No,,,,Yes,,PDF documents,,,,Yes,"Yes – agency has access to source code, but it is not public.",,,,,Yes,,Yes,,Yes,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Potential Applications of AI Models to UAS Post-fire Mapping Data Analysis and Image Processing,Department of the Interior,DOI,OS - Office of the Secretary of the Interior,Mission-Enabling,,None of the above.," Evaluating AI Models for UAS Post-Fire Mapping Analysis Focusing on the Following Issues... Post-Fire Damage Assessment: Efficiently allocate resources for future rehabilitation efforts. Vegetation Recovery Prediction: Assess burn severity to estimate ecosystem regeneration speed. Soil Erosion Risk Mapping: Identify vulnerable areas and guide erosion control measures. Habitat Restoration Planning: Prioritize areas for reforestation, invasive species removal, and soil stabilization. Water Quality Monitoring: Predict sediment runoff, nutrient levels, and contaminants for future water management. Structural Evolution Analysis and Prediction: Analyze fire spread and growth to aid risk assessment and response planning. Environmental Impact Assessment: Aid in planning rehabilitation efforts and assessing ecosystem resilience. Risk Communication and Public Awareness: Inform communities about rehabilitation efforts and safety precautions. Ongoing Post-Fire Rehabilitation Treatment Effects: Assist BAER teams in assessing burned landscapes and monitoring ecological effects. Cross-Disciplinary Integration: Bridge communication gaps between experts in various fields for wildfire risk assessment. Literature Reanalysis: Assist researchers in developing insights, identifying gaps, and informing future research. Assessing Damage and Recovery: Identify fire-damaged objects (e.g., infrastructure, trees) to prioritize rehabilitation. Monitoring Ecosystem Recovery: Identify plant species, soil erosion patterns, and ecological indicators for restoration planning. Identifying Hazards and Safety Risks: Identify post-fire hazards (e.g., unstable terrain, debris) to ensure safety during rehabilitation. Identifying Cultural and Archeological Objects: Identify relevant objects (e.g., artifacts, disturbances) for avoidance during rehabilitation. Customized Rehabilitation Plans: Detect specific objects (e.g., utility poles, contaminated soil) for tailored rehabilitation plans.",see description ,Initiated,Neither,,,,,,,No,,,No,,No,No,Yes,, UAS-derived ,,,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,,,,,,,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Proposal for the I-NEPA System: Leveraging Artificial Intelligence (AI) for Enhanced Efficiency in E,Department of the Interior,DOI,OS - Office of the Secretary of the Interior,Government Services (includes Benefits and Service Delivery),,Creating visually appealing presentations using AI-driven design suggestions.,"I-NEPA aims to streamline and enhance environmental reviews, foster improved public engagement, and facilitate advanced searching, tracking, and reporting of both past and present NEPA actions throughout the Department of the Interior. I-NEPA will leverage the Retrieval Augmented Generation (RAG) process, combining NEPA and policy documents and user input into a large language model (LLM) prompt to get tailored output for each project.",see description ,Implementation and Assessment,Neither,,,,,,,,,,,,,,Yes,,,,,,,,,,,,,,,,,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Public Comment Analysis Tool (PCAT): Leveraging Artificial Intelligence (AI) for Enhanced Efficiency,Department of the Interior,DOI,OS - Office of the Secretary of the Interior,Government Services (includes Benefits and Service Delivery),,Summarizing the key points of a lengthy report using AI.,"Federal agencies struggle with efficiently processing and analyzing public comments, particularly for NEPA actions involving controversial topics with millions of responses. Manual processing is labor-intensive and introduces delays and inconsistencies. PCAT seeks to address these challenges by automating the analysis process steps only to improve accuracy and efficiency.",Streamline environmental reviews and permitting by utilizing AI and ML for efficient and accurate analysis of public comments.,Implementation and Assessment,Neither,,,,,,,,,,,,,,Yes,,,,,,,,,,,,,,,,,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Pilot: Using Machine Learning to Harvest Data for Standardized Species Conservation,Department of the Interior,DOI,FWS,Science & Space,,None of the above.,"The U.S. Fish and Wildlife Service is in the process of reimagining and creating a new database to hold, depict, and disseminate information about the species we manage. This effort will greatly expand and standardize the data related to species and their conservation. We have worked with contractors to develop a preliminary tool that explores the possibiliy of using machine learning and large language modelling to harvest species information related to their biological and physical needs into standardized pick-lists. This includes a way to then vet the harvested data. As part of this we are evaluating the infrastructure needed to containerize tool like this in our environment.","we aim to integrate summmarized data, using some sort of tailored AI and framework or API, into our species workflows in ECOSPhere as it relates to species information. fill in worflows for reporting and analytics",Initiated,Neither,2/14/2024,,,,Developed in-house.,,No,,,No,,No,No,Yes,,Literature provided by field offices related to species information. Evalutaed with sample data taken from published literature in USFWS library Zotero.,,,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,6-12 months,Yes,,Yes,,Yes,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Summarization of documents and output to ECOSphere species workflow,Department of the Interior,DOI,FWS,Science & Space,,None of the above.,"The U.S. Fish and Wildlife Service has a substantial number of documents that we would like to take advantage of NLP or ML process in order to summarize their content. Afterwards, we aim to integrate that summmarized data, using some sort of tailored AI and framework or API, into our species workflows in ECOSPhere as it relates to species information. "," we aim to integrate summmarized data, using some sort of tailored AI and framework or API, into our species workflows in ECOSPhere as it relates to species information. fill in worflows for reporting and analytics",Initiated,Neither,,,,,,,No,,,No,,No,No,Yes,,,,,,Yes,,,,,,,,Yes,,Yes,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Prediction of Suitable Habitat for ESA-listed Species,Department of the Interior,DOI,FWS,Science & Space,,None of the above.,"The U.S. Fish and Wildlife Service currently uses species distribution models (i.e., machine learning algorithms) to predict potential habitat for ESA-listed threatened and endangered species. These habitat predictions are vetted by USFWS field biologists and, in some cases, are used as ranges in IPaC to inform users of possible impacts on ESA-listed species. Species distribution model outputs have also been used to identify suitable habitat on the landscape for potential reintroduction efforts or suitable areas for additional surveys, with the potential to locate new, previously undocumented populations.",fill in worflows for reporting and analytics,Implementation and Assessment,Neither,4/1/2019,4/1/2019,4/1/2019,,Developed in-house.,,No,,,Yes,,No,No,Yes,,Species occurrence data.,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes, ECOSphere,Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Computer Vision Model to Rapidly Identify Habitat Change,Department of the Interior,DOI,FWS,Science & Space,,None of the above.,"The U.S. Fish and Wildlife Service is planning to partner with the Chesapeake Conservancy, an NGO, to develop a computer vision model to monitor for early detection of habitat loss across the landscape, a significant threat to biodiversity, including threatened and endangered species.  By using a computer vision model one can rapidly identify and flag areas where habitat loss may be occurring due to natural or human-caused disturbances.  Early detection can facilitate rapid responses, when appropriate, or allow practitioners to accurately calculate habitat loss over time.  More accurate estimates of habitat loss allow for better management decisions and potentially shorter recovery times for threatened and endangered species."," By using a computer vision model one can rapidly identify and flag areas where habitat loss may be occurring due to natural or human-caused disturbances.  Early detection can facilitate rapid responses, when appropriate, or allow practitioners to accurately calculate habitat loss over time.  More accurate estimates of habitat loss allow for better management decisions and potentially shorter recovery times for threatened and endangered species.",Acquisition and/or Development,Neither,,,,,Developed in-house.,,No,,,No,,No,No,Yes,,Species occurrence data.,,,,,,,,,6-12 months,Yes,,Yes,,Yes,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Applying Deep Learning to Detect and Classify Ocean Wildlife,Department of the Interior,DOI,FWS,Mission-Enabling, ,None of the above.,"Deep learning methods are being advanced to automate data processing and improve the cost-efficiency of remote sensing technologies for surveys covering broad geographic areas and generating very large image datasets. The FWS is partnering with the Bureau of Ocean Energy Management (BOEM), U.S. Geological Survey (USGS), academic institutions, and private contractors to accomplish these objectives. Initial focus has been on marine bird and other wildlife surveys given overlapping agency requirements for these data. A workflow is now being established to advance the imagery data from the sensors, to an AI detection model in the aircraft, to a species classification algorithm that is able to distinguish and count each species, and then a georeferenced location is obtained for each bird. More specifically, a cutting-edge artificial intelligence/deep learning algorithm has been developed to automatically detect seabirds in imagery. This detection algorithm has 80 to 90% accuracy across a wide range of seabird species. In FY25, we will continue to iterate on this model using a human-in-the-loop technique to improve model generalization.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Acquisition and/or Development,Neither,10/1/2016,10/1/2016,On-going development,,Developed with both contracting and in-house resources.,Data not reported by submitter and will be updated once additional information is collected,No, , ,No, ,No,,Yes,,Remote sensing digitial imagery and labels for bird species,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.", , ,Yes,Yes – source code is publicly available.,https://github.com/USFWS/AI-for-USFWS-Migratory-Birds, , ,,No, ,Other,We have local computing resources that are suitable,, ,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Enhancing Migratory Bird Surveys with Thermal Imagery and Deep Learning,Department of the Interior,DOI,FWS,Mission-Enabling, ,None of the above.,"The U.S. Fish and Wildlife Service (USFWS) Division of Migratory Bird Management is aiming to improve upon migratory bird surveys by utilizing aerial remote sensing combined with deep learning (a form of artificial intelligence) analyses to automate survey counts. The goal is to provide accurate wildlife counts while simultaneously reducing risk to pilots by allowing aerial surveys to occur at higher altitudes. In partnership with the College of William and Mary, USFWS has previously demonstrated that thermal remote sensing technology, coupled with deep learning, can provide accurate counts of sandhill cranes (Antigone canadensis) at night during their critically important migratory stopover in the Platte River Valley of Nebraska","Migratory bird spatial locations, bird species identification, and bird counts",Acquisition and/or Development,Neither,8/15/2020,8/15/2020,7/1/2024,,Developed in-house., ,No, , ,No, ,No,,Yes,,Remote sensing digitial imagery and labels for bird species,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.", , ,Yes,Yes – source code is publicly available.,https://github.com/USFWS/AI-for-USFWS-Migratory-Birds, , ,,No, ,Other,We have local computing resources that are suitable,, ,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, SOCS documents analysis,Department of the Interior,DOI,BOEM,Mission-Enabling,,None of the above.," The BOEM Status of the Outer Continental Shelf (SOCS) environmental information portal contains over 900 BOEM documents including Environmental Impact Statements (EISs), Environmental Assessments (EAs), Categorical Exclusion Reviews (CERs), Biological Assessments (BAs), Biological Opinions (BiOPs), National Historic Preservation Act (NHPA) Section 106 documents, policy documents, environmental guidance, reusable content, and environmental studies. Some potential use cases for integrating LLM/AI into SOCS: - Generate resource affected sections for different resources based on existing recent EISs and pull in new information to keep the analysis current. For example, if a wind energy lease is being contemplated in the vicinity of an already analyzed wind energy lease, the information would be pulled from the existing environmental analyses as a starting point, incorporating any new information in the system. The drafted information would include citations and a references cited section in a particular format. - Have LLM/ summarize key findings from multiple studies on similar resources. For example, multiple studies addressing noise or sound impacts to different species could be summarized with references to the source publications.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,9/9/2024,,,,Data not reported by submitter and will be updated once additional information is collected,Data not reported by submitter and will be updated once additional information is collected,No,,,,,,,Yes,,Data not reported by submitter and will be updated once additional information is collected,Data not reported by submitter and will be updated once additional information is collected,,,Yes,"Yes – agency has access to source code, but it is not public.",,,,Less than 6 months,Yes,,Yes,,Yes,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, ONRR Video Hosting Platform [OVHP],Department of the Interior,DOI,ONRR,Mission-Enabling,,Creating visually appealing presentations using AI-driven design suggestions.," ONRR's Video Hosting Platform is a secure, internal-only video hosting site hosting training videos, all-hands meetings, health & wellness, SEP events and more. AI-generation of captions/subtitles, metadata, and chapters to make video hosting, editing, and searching more efficient. AI-generated captions support Section 508 requirements.",see description ,Operation and Maintenance,Neither,,,,,,,,,,,,,,Yes,,,,,,,,,,,,,,,,,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Video Creation and Editing,Department of the Interior,DOI,ONRR,Mission-Enabling,,Creating visually appealing presentations using AI-driven design suggestions.,"To efficiently generate captions, subtitles, text-to-speech, video, photo, and audio.",see description ,Operation and Maintenance,Neither,,,,,,,,,,,,,,Yes,,,,,,,,,,,,,,,,,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, "Predictive AI Applications for Wildlife Monitoring: SeeOtter, a custom built software solution for a",Department of the Interior,DOI,FWS,Science & Space,,None of the above.,"Marine Mammals Management is tasked with Stock Assessments under the Marine Mammal Protection Act, Section 117 and sea otter population monitoring has shifted to image-based surveys, relying on on a YoloV5, AI-assisted process to procure a data set of sea otter observations from photos. Department of Interior agencies, USFWS, USGS, and NPS have all adopted an image-based methodology for sea otter population monitoring, but the model requires continued maintenance and refinement with new surveys, sensor upgrades, and model updates. Unfortunately, there are no AI expert permanent staff tased with this project across Alaska DOI programs and the need has been fulfilled through contractors. SeeOtter was a custom developed software by Collin Power and Evan Wetherington, under contract to USFWS Sea Otter Program, to turn hundreds of thousands of images from across the Gulf of Alaska into datasets usable for statistical analyses by USGS and Stock Assessment Reporting by USFWS and NPS. Glacier Bay NPS assisted in developing an SOP for the AI-assisted processing for sea otter data and Collin Power, under contract with USGS built the system out through a Center for Data Integration Project to make it more accessible to interested users. We would like SeeOtter to be a tool availabel to Partners, across DOI as well as Tribes and researchers, who are interested in applying a predictive AI model for counting objects from imagery. USGS also have plans for large scale photo surveys coming up in 2025/26 and we will be working collaboratively to figure out the support for image processing following the SeeOtter SOP that has been developed in a partnership among USFWS, USGS, and NPS.","The Yolo V5 algorithm provides georeferenced bounding boxes around each object detected in an image, in our case, sea otters with an associated confidence score.",Operation and Maintenance,Neither,,,1/3/2022,,Developed with contracting resources.,Data not reported by submitter and will be updated once additional information is collected,No,,,No,,No,,Yes,,Images captured during aerial sea otter surveys across Alaska.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,"Yes – agency has access to source code, but it is not public.",public release in development.,,,,Yes,,Yes,,Yes,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Non-Generative AI use for Trust Information Analysis and Reporting Tool,Department of the Interior,DOI,BTFA,,,Searching for information using AI.,"The use of Non-Generative AI for identifying blank pages, document classification, title extraction, date extraction. This endeavor is using Azure Computer Vision and Document Intelligence Read to help with identifying blank pages. Azure AI Search Services (This is on hold right now and not listed under AI Tool Name) for document classification, title extraction and date extraction.",see description,Acquisition and/or Development,Neither,,,,,,,,,,,,,,Yes,,,,,,,,,,,,,,,,,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Intelligent Optical Character Recognition,Department of the Interior,DOI,IBC,,,Inputting large amounts of data from paper forms into a digital system using AI.," IOCR is machine learning software that replaces the need for collecting documents and manually entering them into systems. The software learns over time based on volume fed through the software and corrects common errors automatically, such as 0 vs O. Invoice processing is one of the immediate applications considered for IOCR. IOCR will speed the flow of data to the certifying officer for review and ultimate payment. This capability also has broader potential to enhance data extraction, validation, and preservation. IOCR will be deployed in conjunction with Robotics Process Automation to allow for more comprehensive automation solutions. The same robust testing and validation of automations prior to deployment to production will be applied to IOCR."," IOCR will speed the flow of data to the certifying officer for review and ultimate payment. This capability also has broader potential to enhance data extraction, validation, and preservation. IOCR will be deployed in conjunction with Robotics Process Automation to allow for more comprehensive automation solutions. The same robust testing and validation of automations prior to deployment to production will be applied to IOCR",Implementation and Assessment,Neither,,,,,,,,,,,,,,Yes,,,,,,,,,,,,,,,,,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Seasonal/Temporary Wetland/Floodplain Delineation using Remote Sensing and Deep Learning,Department of the Interior,DOI,BOR,Mission-Enabling,,None of the above.,"Reclamation was interested in determining if recent advancements in machine learning, specifically convolutional neural network architecture in deep learning, can provide improved seasonal/temporary wetland/floodplain delineation (mapping) when high temporal and spatial resolution remote sensing data is available? If so, then these new mappings could inform the management of protected species and provide critical information to decision-makers during scenario analysis for operations and planning.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,10/1/2018,,,,Developed in-house.,,No,,,,"Actitivity as proof of concept and did not yield influential scientific information, so IQA not applicable.",No,No,Yes,,"Non-sensitive, mission data describing resource conditions relevant to the condition assessment/prediction question being addressed via AI-ML.",,,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Data Driven Sub-Seasonal Forecasting of Temperature and Precipitation ,Department of the Interior,DOI,BOR,Mission-Enabling,,None of the above.,"Reclamation has run 2, year-long prize competitions where particants developed and deployed data driven methods for sub-seasonal (2-6 weeks into future) prediction of temperature and precipitation across the western U.S. Particpants outperformed benchmark forecasts from NOAA. Reclamation is currently working with Scripps Institute of Oceanography to further refine, evaluate, and pilot implement the most promising methods from these two copmetitions. Improving sub-seasonal forecasts has significant potential to enhance water management outcomes. ",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,12/19/2016,,,,Developed in-house.,,No,,,,"Actitivity as proof of concept and did not yield influential scientific information, so IQA not applicable.",No,No,Yes,,"Non-sensitive, mission data describing resource conditions relevant to the condition assessment/prediction question being addressed via AI-ML.",,,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Data Driven Streamflow Forecasting,Department of the Interior,DOI,BOR,Mission-Enabling,,None of the above.,"Reclamation, along with partners from the CEATI hydropower industry group (e.g. TVA, DOE-PNNL, and others) ran a year-long evaluation of existing 10-day streamflow foreasting technologies and a companion prize competition open to the public, also focused on 10-day streamflow forecasts. Forecasts were issued every day for a year and verified against observed flows. Across locations and metrics, the top perfoming foreacst product was a private, AI/ML forecasting company - UpstreamTech. Several competitors from the prize competition also performed strongly; outperforming benchmark forecasts from NOAA. Reclamation is working to further evaluate the UpstreamTech forecast products and also the top performers from the prize competition. ",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,8/1/2020,8/1/2023,12/1/2024,,Developed in-house.,,No,,,,"Actitivity as proof of concept and did not yield influential scientific information, so IQA not applicable.",No,No,Yes,,"Non-sensitive, mission data describing resource conditions relevant to the condition assessment/prediction question being addressed via AI-ML.",,,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Snowcast Showdown ,Department of the Interior,DOI,BOR,Mission-Enabling,,None of the above.,"Reclamation partnered with Bonneville Power Administration, NASA - Goddard Space Flight Center, U.S. Army Corps of Engineers, USDA - Natural Resources Conservation Service, U.S. Geological Survey, National Center for Atmospheric Research, DrivenData, HeroX, Ensemble, and NASA Tournament Lab to run the Snowcast Showdown Prize Competition. In this competition, particiapnts were asked to develop mehtods to estimate distributed snow information by blending observations from different sources using machine learning methods that provide flexible and efficient algorithms for data-driven models and real-time prediction/estimation. Winning methods are now being evaluated and folded into a follow-on project with NOAA's River Forecast Centers. ",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,4/14/2023,12/7/2021,,,Developed with contracting resources.,Data not reported by submitter and will be updated once additional information is collected,No,,,,"Actitivity as proof of concept and did not yield influential scientific information, so IQA not applicable.",No,No,Yes,,"Non-sensitive, mission data describing resource conditions relevant to the condition assessment/prediction question being addressed via AI-ML.",Data not reported by submitter and will be updated once additional information is collected,,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, PyForecast ,Department of the Interior,DOI,BOR,Mission-Enabling,,None of the above.,Pyforecast is a statistical/ML water supply forecasting software developed by Reclamation that uses a range of data-driven methods. ,Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,1/1/2020,,,,Developed in-house.,,No,,,,"Actitivity yielded forecasting tool that was peer reviewed to the rigor required for influential scientific information, per IQA.",No,No,Yes,,"Non-sensitive, mission data describing resource conditions relevant to the condition assessment/prediction question being addressed via AI-ML.",,,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Improved Processing and Analysis of Test and Operating Data from Rotating Machines,Department of the Interior,DOI,BOR,Mission-Enabling,,None of the above.,"This project is exploring a better method to analyze DC ramp test data from rotating machines. Previous DC ramp test analysis requires engineering expertise to recognize characteristic curves from DC ramp test plots. DC ramp tests produce a plot of voltage vs current for a ramping voltage applied to a rotating machine. By using machine learning/AI tools, such as linear regression, the ramp test plots can be analyzed by computer software, rather than manual engineering analysis, to recognize characteristic curves. The anticipated result will be faster and more reliable analysis of field-performed DC ramp testing.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Acquisition and/or Development,Neither,,,,,Developed in-house.,,No,,,,"Actitivity as proof of concept and did not yield influential scientific information, so IQA not applicable.",No,No,Yes,,"Non-sensitive, mission data describing resource conditions relevant to the condition assessment/prediction question being addressed via AI-ML.",,,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Improving UAS-derived photogrammetric data and analysis accuracy and confidence for high-resolution data sets using artificial intelligence and machine learning,Department of the Interior,DOI,BOR,Mission-Enabling,,None of the above.,"UAS derived photogrammetric products contain a large amount of potential information that can be less accurate than required for analysis and time consuming to analyze manually. By formulating a standard reference protocol and applying machine learning/artificial intelligence, this information will be unlocked to provide detailed analysis of Reclamation's assets for better informed decision making.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,,,Developed in-house.,,No,,,,"Actitivity as proof of concept and did not yield influential scientific information, so IQA not applicable.",No,No,Yes,,"Non-sensitive, mission data describing resource conditions relevant to the condition assessment/prediction question being addressed via AI-ML.",,,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Photogrammetric Data Set Crack Mapping Technology Search ,Department of the Interior,DOI,BOR,Mission-Enabling,,None of the above.,"This project is exploring a specific application of photogrammetric products to process analysis of crack mapping on Reclamation facilites. This analysis is time consuming and has typically required rope access or other means to photograph and locate areas that can now be reached with drones or other devices. By formulating a standard reference protocol and applying machine learning/AI, this information will be used to provide detailed analysis of Reclamation assets for better decision making. ",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,,,Developed in-house.,,No,,,,"Actitivity as proof of concept and did not yield influential scientific information, so IQA not applicable.",No,No,Yes,,Involves use of high resolution imagery of large infrastructure (CUI). ,,,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Effects of vehicle traffic on space use and road crossings of caribou in the Arctic,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"Assessing the effects of industrial development on wildlife is a key objective of managers and conservation practitioners. However, wildlife responses are often only investigated with respect to the footprint of infrastructure, even though human activity can strongly mediate development impacts. In Arctic Alaska, there is substantial interest in expanding energy development, raising concerns about the potential effects on barren-ground caribou (Rangifer tarandus granti). While caribou generally avoid industrial infrastructure, little is known about the role of human activity in moderating their responses, and whether managing activity levels could minimize development effects. To address this uncertainty, we examined the influence of traffic volume on caribou summer space use and road crossings in the Central Arctic Herd within the Kuparuk and Milne Point oil fields on the North Slope of Alaska.","Prediction of liquefaction potential at input site, as it compares to existing case history dataset. ",Implementation and Assessment,Neither,,,10/3/2023,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,"Non-sensitive, mission data describing resource conditions relevant to the condition assessment/prediction question being addressed via AI-ML.","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,No,N/A,None of the above,,No – it is not operationally practical to offer this. (Please explain.),,"No – This AI use case is not subject to the opt-out requirement because the AI functionality is solely used for the prevention, detection, and investigation of fraud or cybersecurity incidents, or the conduct of a criminal investigation.",, Automated Walrus Haulout Monitoring,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"The purpose of this software is to provide a framework for using pre-trained image classification convolutional neural network CNN models to (1) make predictions on unlabled image datasets to provide data for further analysis of walrus (Odobenus rosmarus) coastal haulout occupation and (2) make predictions on image datasets where keywords have been manually (i.e. by human reviewers) added to the image ITPC metadata indicating the content of the images, including the presence and absence of walruses so that the model's performance can be evaluated by comparing its predictions to the human-assigned class labels using confusion matrices and standard classifier evaluation metrics such as accuracy, precision, recall, F1-score (a composite of precision and recall).","screening-level prediction of deformation of embankments, dams, or other facilities due to seismic loading. ",Acquisition and/or Development,Neither,,,2/29/2024,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,"Non-sensitive, mission data describing resource conditions relevant to the condition assessment/prediction question being addressed via AI-ML.","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Forecasting Earthquake Ground Motion Time Series,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"Development of a deep learning models to generate earthquake ground motion time series for potential application to Earthquake Early Warning, Operational Aftershock Forecasting, and the National Seismic Hazard Model.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Acquisition and/or Development,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,"Non-sensitive, mission data describing resource conditions relevant to the condition assessment/prediction question being addressed via AI-ML.","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, CONUS EcoFlows Planning & Prototype,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"To support development of national-scale ecological-flow response models, a benchmark estimate of typical, long-term unaltered flow conditions is needed as a reference benchmark to evaluate flow departures from ""normal"". This model uses unaltered, near-reference USGS gages from across the US, and associated natural landscape datasets, to predict long-term average monthly flows. These models, trained in near-reference sites, are then used to predict the hypothetical near-reference long-term flow conditions of sites with anthropogenic influences, where we have paired biological datasets. By comparing observed flows to these 'natural' expectations, we calculate flow departures from a 'natural normal' and which is used in later ecological modeling.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,10/3/2022,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,"Non-sensitive, mission data describing resource conditions relevant to the condition assessment/prediction question being addressed via AI-ML.","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Prioritized Constituents: Sediment,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"Regional prediction of suspended sediment concentration in unmonitored rivers to characterize sediment transport in the Delaware, Illinois, and Colorado River Basins.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,10/2/2023,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,"Non-sensitive, mission data describing resource conditions relevant to the condition assessment/prediction question being addressed via AI-ML.","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Avian population estimates from passive acoustic monitoring,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,Reliable estimates of avian abundance from acoustic recordings,Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Acquisition and/or Development,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,Avian acoustic recordings,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, FEMA mixed population flood-frequency analysis,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,Classification of historical floods based on causal mechanisms to support improved estimation of flood reoccurrence intervals,Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Acquisition and/or Development,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, "Nutrient, Salinity, and temperature model development",Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"Multiple projects designed to simulate nutrients (phosphorus and nitrate), temperature, and salinity in streams across the U.S. using machine learning approaches.",see description,Initiated,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,"Climate, earth science, land use","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Data-Driven Streamflow Drought,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"Prototype streamflow drought forecasts using data-driven, machine learning approaches for USGS gage locations across the continental U.S.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,10/3/2022,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, AI/ML for aquatic science,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"This project aims to develop novel computational frameworks and AI algorithms for individual fish recognition, by leveraging AI, computer vision and deep learning. The main objectives of this project include: (1) Develop baseline AI models by exploiting visual features and pre-trained deep learning models. (2) Improve individual fish recognition performance, as well as handling new individuals and exploring dynamic environments. (3) Evaluate melanistic markings associated with blotchy bass syndrome to assess the capacity for AI detection of diseased fish. (4) Evaluate deep learning models for individual recognition and respiration rate (ventilate rate) using video data collected in laboratory settings and natural streams.",see description,Initiated,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,Earth Science,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, National-Extent Groundwater Quality Prediction for the National Water Census and Regional Integrated,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"The primary objectives of this project are to (1) provide Nationally consistent predictions of groundwater quality (salinity and nutrients) relevant for human and ecological uses and its influence on surface-water, and (2) develop strategies for integrating these predictions into comprehensive water-availability assessments including the National Water Census and regional Integrated Water Availability Assessments.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,10/1/2021,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,"Earth Science, Land Use, Climate, Water Quality, Population Density","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Use of artificial intelligence tools for optimization and documentation for computer codes,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"For the USGS National Seismic Hazard Model and other earthquake hazards research, computer codes are needed that implement earthquake rupture forecasts and ground-motion models. This project uses ChatGPT to suggest optimizations and documentation for computer codes.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,10/2/2023,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,computer codes,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Improved earthquake detection for research studies,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"Deep learning methods are being used to improve detection of earthquakes to provide more complete, high-resolution catalogs that are used in research to better understand earthquake occurrence, rupture processes and seismic hazard.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Acquisition and/or Development,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,seismic waveform data,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, NGWOS External R&D - Using advanced computing techniques for image-based monitoring,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,Multiple projects conducting research on AI/ML techniques for image-based water monitoring,Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Acquisition and/or Development,Neither,,11/1/2023,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Water Use Model Development,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,Goal to develop process-based and machine learning models to estimate multiple categories of water use across the U.S.,Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,10/1/2021,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,Earth Science,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, National Temperature Observations,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"The objectives of this project are to reduce the burden on Science Centers for the collection, storage, analysis, and processing of quality assurance data with the expectation this will lead to an increase of deployed sensors in the water temperature network. More specifically the project will (1) modify software to allow for processing and storage of discrete water temperature data collected during streamflow measurements, (2) implement workflows and QA checks in data collection software that supports new temperature policies and procedures (3) create a pilot program to support Science Centers in accomplishing 5-pt temperature checks.",see description,Initiated,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,Earth Science,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Downscaling and assimilation of meteorology data for local estimates of irrigation water demand.,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.," We found that in the context of satellite remote sensing for estimates of crop water use, high-frequency and low resolution gridded meteorological reanalysis data contributes about half the total error in our estimates. This is due to well-documented localized effects of terrain and land use that aren't accounted for in meteorological forcing datasets such as NLDAS-2 or ERA-5. Our work suggests that assimilating local meteorological station data alongside land use and terrain information may improve the accuracy of our estimates of local-scale atmospheric demand (i.e., reference evapotranspiration, a function of temperature, wind speed, humidity, and solar radiation). The relationship between the reanalysis products and observed local meteorology appears complex and non-linear. We've found that using deep learning techniques, including Long-Short Term Memory (LSTM) time series modeling along Graph Neural Networks to assimilate local meteorological observations offers promise to improve our local-scale estimates atmospheric demand, and thus irrigated crop water use. We hope to use the information we've gathered to add value to existing meteorological forcing datasets by developing software that reliably downscales low spatial resolution datasets to make more accurate estimates of irrigation water use. Such a product would be especially useful in areas without the highly-developed meteorology and climatology datasets that are exclusive to the United States, such as PRISM and GridMET.",see description,Initiated,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, National-Extent Groundwater Quality Prediction for the Integrated Water Availability Assessments,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"The primary objectives of this project are to (1) provide Nationally consistent predictions of groundwater quality (salinity and nutrients) relevant for human and ecological uses and its influence on surface-water, and (2) develop strategies for integrating these predictions into comprehensive water-availability assessments including the National Water Census and regional Integrated Water Availability Assessments.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Building capacity for assessment and prediction of post-wildfire water availability,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,Model development to forecast water quality impacts of wildfires in the western U.S. focused on suspended sediment and salinity,see description,Initiated,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,Earth Science,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Mapping sagebrush from drones to satellites,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"Vegetation maps are critical tools for that are widely used in applications including habitat modeling, evaluating effectiveness of habitat restoration, and understanding the ecological implications of biological invasions. We are using machine learning and imagery from unmanned aerial vehicles (UAV), aircraft, and satellites to extend presence modeling to map fractional cover of sagebrush in the Dakotas, where accurate maps of sagebrush are needed to identify seasonal habitats of sage-grouse for the Bureau of Land Management.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Acquisition and/or Development,Neither,,6/7/2023,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Delineating sub-surface drainage using satellite imager,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"Knowing subsurface drainage (tile-drain) extent is integral to understanding how landscapes respond to precipitation events and subsequent days of drying, as well as how soil characteristics and land management influence stream response. Consequently, a time series of tile-drain extent would inform one aspect of land management that complicates our ability to explain streamflow and water-quality as a function of climate variability or conservation management.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Operation and Maintenance,Neither,,,9/30/2023,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,"A UNet machine-learning model, a convolutional neural network designed to highlight objects of interest within an image, to delineate tile-drain networks in panchromatic satellite imagery without additional data on soils, topography. Satellite imagery, soils data","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Vegetation mapping on the Hawaiian island of Lanai,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"Creating high-resolution vegetation mapping approach that combines satellite imagery, machine learning, and expert knowledge to accurately classify plant species across the Hawaiian island of Lanai, producing detailed maps that can support conservation planning and monitoring of both native and invasive species.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,Digital Globe WorldView-2 satellite imagery; airborne imagery collected by EagleView,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Machine Learning for automatic fracture mapping and rock identification,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,Machine learning algorithms are being used to improve detection and characterization of faulting after major surface rupturing earthquakes and identify and collect imagery of fragile geologic features with application to assessment of earthquake hazards.,Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Acquisition and/or Development,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,Geological field data,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Reinforcement Learning for Helmholtz Coil Operation and Simulation,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"The USGS is using AI/ML to optimize performance of its magnetic observatories. For example, reinforcement learning (RL) can significantly aid in the operation of a Helmholtz coil by optimizing its performance in generating uniform magnetic fields.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Acquisition and/or Development,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Deep-learning Integration into NEIC Operations,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,The National Earthquake Information Center (NEIC) is improving its earthquake detection and processing systems by leveraging artificial intelligence and machine learning.,Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Predictions of PFAS Concentrations in Groundwater,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"A model of PFAS occurrence in groundwater at the depths of public and private drinking water supplies across the CONUS has been created and is expected to receive national attention. The model predicts occurrence of PFAS (detect or non-detect) in 1x1 km grid cells, and leverages the best data available in 2023. By updating this model, the goals are to (1) provide concentration predictions, instead of occurrence predictions, and (2) improve model accuracy owing to a larger set of sample data to train the model with. Concentration estimates would allow for better leveraging of resources to areas with predicted significant impacts.",see description,Initiated,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,"Well Data, water quality","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, NGWOS External R&D - Using advanced computing techniques for mobile monitoring platforms,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,This project is designing a networking approach for underwater robotic sensor platforms so that they can work together without direct human intervention to adjust their monitoring operations to changing bathymetry and water conditions using artificial intelligence algorithms and underwater communications.,Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Acquisition and/or Development,Neither,,11/1/2023,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,Earth Science,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Population and critical habitat modeling of overwintering monarch butterflies,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"Monarch butterflies in the western United States overwinter at very specific locations across coastal California. As monarch population decline it become important to identify the characteristics of what makes an overwintering grove a suitable habitat. Understanding the land cover and climatic factors that influence site selection by monarch can aid land managers in both making decisions to support exisiting critical habitat, and identify previously unknown locations where monarchs overwinter",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,10/1/2023,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,"High resolution land cover data, population abundance data, regional climate data","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Automating the Detection and Classification of Wildlife in Aerial Imagery,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"The US Geological Survey (USGS), Bureau of Ocean Energy Management (BOEM), and US Fish and Wildlife Service (FWS) are partnering on a multi-year effort to develop deep learning algorithms and tools for the detection and classification of seabirds and other marine wildlife in aerial imagery. The tools and workflows developed by this project will be used by BOEM to assess wildlife populations as part of planning and monitoring for offshore energy development.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,10/1/2018,,10/1/2018,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,"High-resolution aerial imagery, expert annotations","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Machine Learning algorithm for stream velocity prediction,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,The algorithm will be used to develop and incorporate a time-of-travel web-based application that will allow users to estimate travel times in a spill response scenario with greater accuracy.,Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Acquisition and/or Development,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,streamflow data,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Machine Learning for streamflow forecasting,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"In the Pacific Northwest, where the Willamette River is located, floods are becoming more common and severe. To help prevent flooding, the area has 13 dams built to control floods. This project, a collaboration with Portland State University, aims to build a smart system to predict river flows accurately and quickly across the Willamette River Basin by using state-of-the-art statistical and computational models as well as new hydrologic observations. The project aims to improve our understanding of how to use new river flow observational data that USGS provides to make better predictions.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Acquisition and/or Development,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,streamflow data,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Automated otolith aging using image processing,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"Fisheries managers and researchers often need to know the age of fish for population estimates, stock assessment, and similar projects. Fish otoliths (an ear bone) often accumulated rings annual (similar to tress). This process traditionally is done manually and can vary across individual agers. We are training an image process ML program to automate this process to see if we can reduce variability across individual agers and automate the aging process of counting otolith rings possibly saving time.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,Otolith images (pictures) with known ages,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Automating blood smear cell counts using machine learning,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"We exposed largemouth bass to an immunogen (poly I:C) to stimulate an antiviral response and collected blood smears from 38 of these fish. The blood smear slides were scanned at 83x, digitized using an Aperio ScanScope CS, and uploaded for labeling using SageMaker Ground Truth software. We are working with USGS Cloud Hosting Solutions to label training sets of images that include lymphocytes, monocytes, and granulocytes. We are in the process of validating and testing the ability of the model to accurately recognize and distinguish WBC images. To accomplish this, we are comparing manual cell counts among the 3 human readers, manual cell counts to automated cell counts by the model, and automated cell counts of novel tiles taken from the training slides.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,10/1/2023,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,"blood smear slides were scanned at 83x, digitized using an Aperio ScanScope CS, and uploaded for labeling using SageMaker Ground Truth software","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Machine learning for tsunami source zones,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,State of the art tsunami hazard analysis for coastal communities and infrastructure is computationally demanding. Ml will be used to select the most representative source zones (among thousands of offshore earthquake ruptures),Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,10/1/2024,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,Offshore fault slip rate data and historical seismicity,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Shoreline Modeling,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"Exploring the prospect of using AI/ML models to predict shoreline change and compare accuracy to traditional models. Examples of AI/ML models include LSTM, CNN, and Transformers",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Determining the resource potential of critical minerals in seafloor massive sulfide deposits,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,ML to input data on seafloor massive sulfide geochemstry to predict composition.,Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,Geochemical data,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, "Oceanographic, coastal, and geomorphic change analysis: data generation, QC/QA, and data management",Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"Machine learning to quantify coastal/marine change across broad scales. QC/QA processes in place to assess data robustness. Verified data will be used by USGS projects for forecasting trends (ie, shorelines, role of permafrost) in a variety of coastal/marine settings for US coasts.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,10/1/2024,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,"Deep Learning based image segmentation, time-series methods, classification & regression methods. Custom scripts/tools using python/pytorch/tensorflow/keras and other ML libraries and APIs","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Synthesizing mapping and monitoring data to inform prairie dog management in National Parks,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"Synthesizing Mapping and Monitoring Data to Understand Fluctuations in Prairie Dog Colony Size and Densities in Theodore Roosevelt National Park. There have been efforts to capture variation in the size and extent of prairie dog colonies at Theodore Roosevelt National Park in some form since the 1940s. Prairie dog colonies have been mapped semi-annually since the 1990s, but shifting priorities and a largely static budget have made it difficult for park staff to continue mapping. Furthermore, little research has been conducted with the existing mapping data to assess prairie dog habitat quality and the factors that affect colony size fluctuations. This project is aimed at developing more cost-efficient methods for prairie dog colony mapping (that is, remote sensing techniques) and developing indices and models that can help managers derive population inferences based on colony area. Park staff need modeling and remote analysis tools they can use to assess variability in prairie dog populations within the park. These tools should include a method for remotely assessing prairie dog colony size and a predictive tool that can help park managers understand the relationship between colony size and prairie dog population size.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,10/1/2020,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,Imagery on prairie dog colony and population size,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, "Quantifying the effects of land-use change and bioenergy crop production on pollinators, wildlife, a",Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"The annual migration of monarch butterflies in North America represents a biological phenomenon unique to our planet, covering more than 4,000 kilometers and requiring multiple generations of monarchs to complete. The monarch was proposed for listing under the Endangered Species Act in 2014 because of significant population declines and extinction risk. Disappearance of milkweed, the essential host plant for monarch larvae, has been implicated in the decline of the eastern monarch population. The objective of this study is to test the effectiveness of using uncrewed aircraft systems (UAS) and artificial neural networks to quantify the density of common and showy milkweed in working grasslands of Minnesota and North Dakota. First, NPWRC scientists will develop a machine learning algorithm for detecting milkweeds from UAS-collected aerial images. Second, NPWRC will validate the algorithm by comparing plot-level counts of milkweed estimated from UAS images to field count data across a range of milkweed densities. Lastly, NPWRC will take steps towards facilitating the integration of this technology into the Integrated Monarch Monitoring Program by estimating the number of spatially independent UAS images required for achieving accurate and precise estimates of milkweed across entire fields. In fiscal year 2019, preliminary UAS flights were completed; however, this project was put on hold because of a DOI ruling that grounded all UAS flights. No USGS UAS flights will be completed until this ruling is changed.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,10/1/2018,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,UAS images,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Computationally efficient emulation of spheroidal elastic deformation sources using machine learning,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"Elastic continuum mechanical models are widely used to compute deformations due to pressure changes in buried cavities, such as magma reservoirs. In general, analytical models are fast but can be inaccurate as they do not correctly satisfy boundary conditions for many geometries, while numerical models are slow and may require specialized expertise and software. To overcome these limitations, we trained supervised machine learning emulators (model surrogates) based on parallel partial Gaussian processes which predict the output of a finite element numerical model with high fidelity.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,1/2/2023,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,model output,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Wildlife species recognition and distance from camera estimation,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,Reliable population estimates of animal density.,Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Acquisition and/or Development,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Automating blood smear cell counts using machine learning,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"We exposed largemouth bass to an immunogen (poly I:C) to stimulate an antiviral response and collected blood smears from 38 of these fish. The blood smear slides were scanned at 83x, digitized using an Aperio ScanScope CS, and uploaded for labeling using SageMaker Ground Truth software. We are working with USGS Cloud Hosting Solutions to label training sets of images that include lymphocytes, monocytes, and granulocytes. We are in the process of validating and testing the ability of the model to accurately recognize and distinguish WBC images. To accomplish this, we are comparing manual cell counts among the 3 human readers, manual cell counts to automated cell counts by the model, and automated cell counts of novel tiles taken from the training slides.",see description,Initiated,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,"blood smear slides were scanned at 83x, digitized using an Aperio ScanScope CS, and uploaded for labeling using SageMaker Ground Truth software","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Machine Learning to evaluate water quality,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,Examining the effect of physicochemical and meteorological variables on water quality indicators of harmful algal blooms in a shallow hypereutrophic lake using machine learning techniques.,Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,9/30/2024,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,water-quality data,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Ecological niche models for bat species,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,We are trying to understand what environmental factors determine the presence and absence of bat species across their range.,Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,1/1/2022,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,"bat presence locations, environmental raster data","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, HotLINK: The volcanic hotspot learning and identification network,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"An increase in volcanic thermal emissions can indicate subsurface and surface processes that precede, or coincide with, volcanic eruptions.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Operation and Maintenance,Neither,,,10/1/2023,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,MODIS and VIIRS infrared satellite imagery from NASA and NOAA,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Development of a Strategic Framework for Use and Implementation of Machine Learning in Energy Resour,Department of the Interior,DOI,USGS,Mission-Enabling,,Improving the quality of written communications using AI tools.,"The overarching objective for this project is development of a strategic framework for integrating ERP science with traditional information technology related platforms. The proposed framework includes (1) adoption of ML pipelines/models in ERP project workflows, tailored to specific needs of ERP project scientists; (2) modernization of key ERP data assets through API extension to meet the needs for data accessibility in big data platforms, analytics libraries, and machine learning models; and (3) technology transfer, targeted training, and multi-disciplinary career development for existing geospatial ERP workforce. Adoption of the proposed strategic framework will help us achieve this 21st century science vision and position the ERP to more effectively deliver its unique data-driven information products.",see description,Acquisition and/or Development,Neither,,,,,,,,,,,,,,Yes,,,,,,,,,,,,,,,,,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, "Quantifying Watershed Controls on Fine Sediment Flux to Lake Tahoe, California/Nevada",Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"The variability in precipitation state impacts the fine sediment (FS, < 16 um) flux from upland areas to Lake Tahoe influencing lake clarity. We used supervised random forest regression models to estimate watershed parameters of importance that drive sediment flux.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,10/1/2019,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,"Stage and turbidity from NWIS, water balance variables from Western Land Data Assimilation (NASA) land surface model.","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Seismology of Magmatic Injection,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,USGS staff are working with a student from Baylor University to use machine learning and network covariance to o understand the nature and dynamics of seismic sources associated with magmatic injection and magmatic transport. This information is necessary to understand volcanic systems. Seismic investigations are also being done for magma plumbing.,Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,10/1/2023,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Earthquake Catalog Development,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"Using AI/ML to develop more complete and robust earthquake catalogs, including focal mechanisms. This includes volcanic earthquake catalog enhancement using integrated detection, matched-filtering, and relocation tools.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,10/1/2021,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,Seismic data collected by HVO during a nodal ,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Seedling Identification and Percent Growth Analysis,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"his project aims to automate the extraction of alphanumeric labels and analyze seedling growth in petri dish images. Labels identifying seed type, treatment, and replication are extracted using Optical Character Recognition (OCR), saving time and reducing human error. Additionally, k-means clustering is applied to segment seedlings from the background, enabling quantification of percent growth over time. The process addresses challenges such as varied image orientations, lighting condidtions, and label placements. By automating label extraction and seedling measurement, the workflow accelerates data analysis, improves accuracy and supports scalable environmental and toxicological research.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,10/1/2023,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,Numerous images of seedlings taken over a span of 5 days. Around 2000 images in total.,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Gulf Coast Geologic Energy Machine Learning,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"Machine learning has been used to predict expected ultimate recovery of shale oil wells in a previously studied assessment unit. These predictions use multi-layer perceptron based artificial neural networks along with decline-curve based estimated ultimate recoveries and a feature database of geological, reservoir, and well completion parameters. Researchers are developing a ML model using elemental data to predict total organic carbon.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,10/1/2023,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Flow Photo Explorer to estimate flow,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.," Full USGS streamgages can be expensive for cooperators, especially on internment streams, so a lower cost, low maintenance method for determining if a stream is flowing and relative flow volumes is needed.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,"Flow Photo Explorer field photos","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Predicting Sparse (Geothermal) Resources Availability by using Machine Learning,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"This research is developing the machine learning (ML) tools (a subset of artificial intelligence) to predict the availability of sparse natural resources (e.g., geothermal, minerals) at regional levels by providing careful consideration to mathematical and geostatistical practices that adhere to geoscience processes. The associated challenges include developing new ML metrics for evaluating model performance that work with sparse natural resources, addressing the extreme mathematical sparsity of these resources at the regional scale, and engineering new evidence layers to inform modeling workflows. The goals of this work include increasing the explainability, reproducibility, and accessibility of the assessment modeling process.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Acquisition and/or Development,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,Data on geothermal and mineral resource locations,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, National Wildlife Disease Database,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"The National Wildlife Health Center (NWHC) has contracted the Pacific Northwest National Laboratory (PNNL) to build a national wildlife disease database (NWDD). Funded through the American Rescue Plan Act of 2021 (ARPA), the NWDD will bring together various wildlife health data streams across informational domains (i.e., laboratory results, environmental observations, news media, etc.) to provide situational awareness and advanced analytics to natural resource authorities around the country. As part of the NWDD, PNNL plans to integrate their Canvas software. Canvas is a machine learning, AI, and data science tool that can visualize and contextualize information from one or more sources.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Acquisition and/or Development,Neither,,9/30/2024,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,"Wildlife morbidity & mortality reports, wildlife disease surveillance, environmental observations, media","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Landform Mapping GeoAI,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"NASA is helping the USGS is developing a faster, smarter process for creating a national landform dataset using machine learning and active learning techniques. These tools help prioritize which data experts should label, saving time and improving accuracy, especially for rare or complex features. Since the data involves 3D shapes, the project is exploring advanced methods like topological analysis and deep learning to better analyze the data. A basic user interface will also be developed to make labeling easier and lay the foundation for future tools like natural language queries.",see description,Initiated,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,"elevation data and derivatives, geographic names and location, topographic maps Ai Tools: pytorch, R, tensorflow libraries","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Using Machine Learning Methods for Automatic Discovery and Catalog of North Dakota Stock Ponds and Other Impoundments,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"The purpose of this project is to address the challenges in cataloging stock ponds and other impoundments by developing a machine learning (ML) workflow that processes satellite imagery for the extraction of their location and type. This workflow will be specifically designed to distinguish between multiple types of stock ponds, stock dams and dugouts, and other impoundments. The second goal is to utilize this ML workflow to build a new, comprehensive catalog of stock ponds and other impoundments. This catalog will be unique in its inclusion of smaller stock ponds, which are often omitted in existing databases, thereby offering a more accurate and complete understanding of stock pond and impoundment distribution and types.",see description,Initiated,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,"LiDAR data for the state of North Dakota AI Tools: UNets","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Improved point cloud classification of 3DEP lidar data using Deep Learning models,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"Point clouds are remote sensing data that is represented by points in space that can represent earths surface. The class of the point can identify the surface as belonging to buildings, trees, or roads for example. This study is testing methods for enhancing 3DEP lidar point data classification to include more surface types at a finer resolution. The tests are using deep learning (DL) models to refine and enrich the 3DEP data classification.",see description,Initiated,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Knowledge Graph development,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,Research and development of knowledge graph designs to make information more interoperable based on logical relationships automatically identified from text. The research work includes using AI for natural language processing.,see description,Initiated,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,"Domain knowledge bases or ontologies for each use case (names, elevation, points of interest, etc.) AI Tools: ESRI Knowledge","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Predicting inundation dynamics of small forested wetlands,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,This project aims to help land managers in the Upper Midwest understand the wetting/drying dynamics of small wetlands relevant to amphibians. The approach will leverage field observations of wetland water levels as training data in a random forest model.,Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Acquisition and/or Development,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Hydrograhy feature extraction from remotely sensed data,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"Surface water features shown on maps, also known as hydrography, show the available water that can greatly influence the quality of life and the environment. Mapping hydrography is a complex task because the amount and distribution of surface water continuously varies with weather conditions. This research is testing machine learning techniques to develop models that predict the location of surface water from remotely sensed elevation and image data. Machine learning techniques are methods that train a model based on existing hydrography features in training area. After training a model, it can be applied to other areas where hydrography data do not exist, and the model can potentially be applied to new remote sensing data to help update hydrography features over time.",see description,Initiated,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,"high resolution elevation data, imagery","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Using machine learning to detect invasive bullfrogs,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,Detecting bullfrogs along their invasion front in order to inform removal efforts,Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,5/1/2020,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,audio recordings,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, "Deep Learning application for automated mapping of surficial landforms, surficial geological deposit",Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,Florence Bascom Geoscience Center (FEDMAP-funded): The Bascom Geoscience Center (FEDMAP) is using the deep-learning capabilities implemented within the ESRI ArGIS Pro software platform to automate the process of mapping surficial landforms and related surficial geologic deposits from lidar-derived topography. We are also using this capability to train models that can identify abandoned mine sites from lidar-derived topography.,Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,10/1/2024,10/1/2024,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,Published geologic maps and user-created data,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Zero shot segmentation to expedite Quaternary geologic mapping,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,Geosciences and Environmental Change (GEC) Science Center (FEDMAP-funded): The construction of detailed geologic maps requires a lot of manual GIS data input to outline the extent of interpreted geologic features. We are working to extend and adapt the Segment Anything Model for identification of Quaternary geologic features in order to expedite the process of creating GIS data for geologic maps.,Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Acquisition and/or Development,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Oil Spil Response for Ice-Covered Rivers,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"This project uses backscatter data from the SAR (synthetic aperture radar instrument) on the Sentinel-1 satellite to determine ice phenology for large rivers and lakes based on machine learning method (i.e., a Random Forest Classifier). The goal of this DOI Inland Oil Spill Preparedness Program (IOSPP) funded work is to provide rapid, near real-time information to oil spill response crews concerning about the safety of ice-covered areas (FY2023-25).",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Operation and Maintenance,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Modeling Sediment Abundance in the Eastern Snake River Plain Aquifer Using Supervised Learning,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,This project aims to model sediment abundance in the eastern Snake River Plain aquifer near the Idaho National Laboratory. We are employing supervised learning techniques to predict sediment presence at various depths within boreholes using natural gamma ray readings. The model is trained with independently derived estimates of sediment probability to enhance its accuracy. This initiative will significantly aid scientists in understanding and managing sediment distribution in the aquifer.,see description,Initiated,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Pacific Northwest Stream Flow Permanence,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"We are using machine learning approaches, specifically the random forest algorithm, to provide spatially explicit estimates of the presence of year-round surface flow in streams across large (several US states) geographic extents. Empirical random forest models include discrete flow/no flow observations as the response variable and a broad suite of physio climatic covariates. The models are used to inform management decisions that require streamflow classification of perennial versus non-perennial which is the charge of many land steward agencies including the Bureau of Land Management, U.S. Forest Service and State and private forests. Funding for this project is congressional allocated funds (FY2023-25).",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Operation and Maintenance,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, SAMPLE Toolbox,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,A toolbox for land managers to develop plans for monitoring vegetation,Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Acquisition and/or Development,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Mapping wildfire fuels in previously burned landscapes,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,The goal is to understand how land management treatments affect the probability of reburning.,Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Acquisition and/or Development,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Inventorying landforms with convolutional neural networks,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"Geosciences and Environmental Change (GEC) Science Center (FEDMAP-funded): LiDAR derived topographic data images earth's surface in unprecedented detail and is available across most of the country. This data reveals landforms relevant for understanding many scientific phenomena (e.g., patterned ground) and natural hazards (e.g., karst). We are developing simple pipelines to train and deploy convolutional neural networks on high resolution topographic data in order to efficiently identify and inventory these features.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Lava lake thermal pattern classification using self organizing maps and relationships to eruption pr,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,We apply a machine learning algorithm called self-organizing maps (SOM) to thermal infrared time-lapse images .,Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,1/1/2024,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Advancing image-based surveys to support sea duck conservation along the Pacific Flyway,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"For most of their annual cycle, North American sea ducks are densely distributed in estuaries and along the coastal nearshore where they are susceptible oil spills, energy development, changing ocean conditions, and other potential threats. Observer-based aerial surveys have been an important tool for evaluating coastal distributions and estimating population abundances to understand sea duck responses to their changing environment. However, safety, expense, observer bias and lack of methodological consistency are rising concerns associated with observer-based surveys, making it imperative to transition to more sustainable methods. Digital aerial surveys (DAS) that automate counts from aerial imagery using convolutional neural network (CNN) models are one way to improve survey safety and count accuracy. We are developing a standardized DAS for the lower Pacific Flyway to help maximize safety, while improving data consistency and model accuracy among important regions within the Flyway.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Acquisition and/or Development,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Probabilistic source classification of large tephra producing eruptions using supervised machine lea,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.," USGS has produced a model that accurately and confidently identifies large tephra-producing eruption volcanic sources in the Alaska-Aleutian arc using only in situ geochemistry. The model is a voting ensemble classifier comprised of six conceptually different machine learning algorithms trained on proximal tephra deposits that have had their source positively identified. Eruptive products from Alaska's Aleutian Arc-Alaska Peninsula and Wrangell volcanic field were used as a test environment for 11 supervised classification algorithms, trained on nearly 2000 electron probe microanalysis measurements of glass major oxides, representing 10 volcanic sources.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, InSAR and other geodetic studies at Volcanoes,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,The USGS uses InSAR (Inferometric Synthetic Aperture Radar) to map ground deformation and track volcanic activity globally. Artificial Intelligence approaches are being used to recognize transient signals in combined InSAR and GPS data that may be indications of impending hazardous volcanic activity.,Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Climate Futures for Lizards and Snakes in Western North America,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,Identifying new management challenges to reptiles based on shifting environmental conditions,Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Operation and Maintenance,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Predicting inundation dynamics of small forested wetlands,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,This project aims to help land managers in the Upper Midwest understand the wetting/drying dynamics of small wetlands relevant to amphibians. The approach will leverage field observations of wetland water levels as training data in a random forest model.,Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Acquisition and/or Development,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Intelligent National Map project,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"The Intelligent National Map is a vision by the U.S. Geological Survey (USGS) to make mapping smarter by using advanced technology like artificial intelligence (AI). This project aims to create maps that can update themselves automatically, detect changes in the environment, and provide better, faster information for decisions about land, water, and natural resources.",see description,Initiated,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Machine-learning model to delineate sub-surface agricultural drainage from satellite imagery,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"We trained a UNet machine-learning model, a convolutional neural network designed to highlight objects of interest within an image, to delineate tile-drain networks in panchromatic satellite imagery without additional data on soils, topography, or historical tile-drain extent. This was done by training the model to match the accuracy of human experts manually tracing the surface representation of tile drains in satellite imagery. Our approach began with a library of images that were used to train and quantify the accuracy of the model, with model performance tested on imagery from two areas that were not used to train the model. Satellite imagery included acquisition dates from 2008 to 2020. Training imagery was from agricultural areas within the US Great Lakes basin. Validation imagery was from the upper Maumee River, tributary to western Lake Erie, and an Indiana, Ohio-River headwater tributary. Our analysis of the satellite imagery paired with meteorological and soil data found that during spring, a combination of relatively high solar radiation, intermediate soil-water content and bare fields enabled the best model performance. Each area of interest was heavily tile-drained, where better understanding the movement of water, nutrients, and sediment from fields to downstream water bodies is key to managing harmful algal blooms and hypoxia. The trained UNet model successfully identified tile drains visible in the validation imagery. https://doi.org/10.1002/jeq2.20493",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Environmental streamflows in the United States: historical patterns and predictions,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.," It is important that environmental streamflow assessments by water managers consider changes in climate, land use, and water management; this cannot be done effectively without understanding historical variability and changes in environmental streamflows. Estimates of environmental streamflows also are needed for ungaged streams and machine-learning methods are likely useful for this. We are analyzing historical change and variability at hundreds of streamflow gages across the United States for a suite of environmental streamflows and using machine-learning methods to estimate environmental streamflows for thousands of ungaged stream reaches.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Acquisition and/or Development,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, PROSPER,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above., Machine learning approach to estimating the annual probability of streamflow permanence at a sub-reach (10m) scale.,Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Operation and Maintenance,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, [Un]supervised clustering of [non-]earthquake signals commonly recorded on regional seismic networks,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"Surficial mass movements (SMMs), such as landslides and rockfalls, have seismic signatures distinct from other routinely recorded seismic sources like earthquakes and explosions. However, overlaps between the characteristics of these signals can still make it difficult to discriminate between source types during operational seismic monitoring. This ambiguity motivates the development of automated techniques for seismic signal classification. Furthermore, seismic differentiation within the broad class of SMMs has additional scientific and rapid response value. Examination of SMM seismic waveforms highlights a particularly strong contrast between the signals produced by primarily vertical processes versus processes that have a non-negligible horizontal component such as landslides and avalanches. A machine learning (ML) classification scheme is being investigated for differentiating between seismic signals generated by falls versus slides. Additionally included are shallow earthquakes and blasts because these are most similar to SMM signals and are commonly recorded on regional seismic networks. These classes, therefore, are the most useful for automated classification. Test datasets derive from waveform picks in the Exotic Seismic Events Catalog, a diverse collection of non-earthquake seismogenic surface events, and the Advanced National Seismic System (ANSS) Comprehensive Earthquake Catalog. In development is a shallow, feature-based approach to classification using statistical metrics extracted from waveforms. Feature importance metrics provide insight into the ML method, which leverage both unsupervised techniques and supervised techniques.",see description,Initiated,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, "Extracting robust, searchable data from narrative geologic descriptions",Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"Geosciences and Environmental Change (GEC) Science Center (FEDMAP-funded): Geologic map units are often described in narrative text descriptions. These descriptions sometime contain useful data (e.g., about lithology, unit thickness), but it is hard for users to operationalize that information since is not codified in any standard way. We are using natural language processing and large language models to parse readily queryable information out of thousands of descriptions in a national database of geology.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,"Descriptions of geologic units taken from published reports. ","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Classifying GPS data to understand flight behavior of birds.,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,This project will help understand under what circumstances eagles are more likely to collide with wind turbines.,Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Whole-lake indexing of round goby abundances with photographic catch data,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"USGS is responsible for monitoring abundances of prey fish across the entirety of the Great Lakes. This project utilizes autonomous vehicles and artificial intelligence to quantify abundances of one of the most abundant prey fishes in the Great Lakes, an invasive species called Round Goby. Robotic surveys of Roung Goby are carried out across three of the five Great Lakes each year. The work also characterizes surface geology, which influence round goby abundances.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Predicting PFAS in shallow soils in northern New England,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"This project leverages statewide shallow soil data and machine learning methods (boosted regression tree models) to predict PFAS in soils across Maine, New Hampshire, and Vermont.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Improving accuracy and precision of sonar-based estimates of fish abundance,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"USGS uses sonar to monitoring prey fish populations to support fisheries management decision making for the eight US Great Lakes states, Fish and Wildlife Service, and numerous tribes. Sonar-based estimates of fish abundance are prone to inaccuracies that can limit their utility. New technologies are being strategically cultivated to improve the accuracy and precision of USGS's annual prey fish abundance estimates. Artificial intelligence is being used to accelerate data processing.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,Sonar transect data collected by conventional vessels and uncrewed surface vehicles,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Classifying GPS data to understand flight behavior of birds.,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,This project will help understand under what circumstances eagles are more likely to collide with wind turbines.,Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Machine learning-based landscape feature classification using satellite and airborne imagery,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,We use existing machine learning classifiers on satellite and airborne imagery to enhance the accuracy of habitat and land cover classifications.,Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Predicting PFAS occurrence in groundwater using machine learning,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"This project leverages USGS groundwater data and machine learning methods (boosted regression tree models) to predict PFAS occurrence in groundwater at the depths of drinking water supplies across the conterminous U.S. A paper with the results of this effort was published in early FY25. A second iteration of this project will aim to predict concentration ranges instead of only occurrence, and commenced at the beginning of FY25.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Machine Learning Image Classification,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"The mission of the PLACE (Patterns in the Landscape - Analyses of Cause and Effect) project is to inform land managers, planners and researchers about historical and current changes to human and natural environments. PLACE focuses on floods, droughts, and fires, which are increasing in severity, extent and frequency around the globe. The PLACE project utilizes random forest machine learning to classify wetlands and soil moisture at large scales, and to quantify causal processes behind wildfire.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Zero shot segmentation to expedite Quaternary geologic mapping,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,The construction of detailed geologic maps requires a lot of manual GIS data input to outline the extent of interpreted geologic features. We are working to extend and adapt the Segment Anything Model for identification of Quaternary geologic features in order to expedite the process of creating GIS data for geologic maps.,Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Acquisition and/or Development,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Inventorying landforms with convolutional neural networks,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"LiDAR derived topographic data images earth's surface in unprecedented detail and is available across most of the country. This data reveals landforms relevant for understanding many scientific phenomena (e.g., patterned ground) and natural hazards (e.g., karst). We are developing simple pipelines to train and deploy convolutional neural networks on high resolution topographic data in order to efficiently identify and inventory these features.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Tracking wetlands and water movement across watersheds,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"Accurate prediction of flood and drought impacts requires understanding upstream surface water storage dynamics and storage capacity. We use machine learning algorithms to classify satellite imagery into open and vegetated water extent. We then use machine learning and deep learning algorithms to relate daily river discharge to meteorology and surface water storage dynamics. This work is funded by USEPA, Office of Research and Development.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Acquisition and/or Development,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, "Everglades-Flux, Digital Surveys",Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,We are creating a program that can automatically process Normalized Difference Vegetation Index images and come up with a true value of live vegetation. It will also gap fill missing data via AI/ML programs.,Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Acquisition and/or Development,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Cooperative Agreement Oakridge National Lab-USGS Research: Using Artificial Intelligence to Improve,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"The research objectives of this project will focus on developing workflow processes and tools that will form the framework for implementation of data management and improvements to data management lifecycles. This includes removing dependencies on personnel to perform manual tasks that enable discovery, access, and formatting of potentially desirable data to users, and instead automate the long-term management of those data to ensure that they are readily usable under various automation scenarios. Specific objectives of this project aligned with AI use cases include: 1. Research and design an algorithm to characterize and visualize data contents in USGS data repositories and DOE ARM holdings by adapting AI and machine learning approaches. 2. AI Tool development to help scientists generate standardized metadata (ISO/FGDC) 3. Using AI principles, automate methods for performing FAIR (Findable, Accessible, Interoperable, and Reusable) data assessments using training data developed for USGS's State of the Data Rubric",see description,Initiated,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Improved earthquake detection for research studies,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"Deep learning methods are being used to improve detection of earthquakes to provide more complete, high-resolution catalogs that are used in research to better understand earthquake occurrence, rupture processes and seismic hazard.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Acquisition and/or Development,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Machine learning for tsunami source zones,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"State-of-the=art tsunami hazard analysis for coastal communities and infrastructure is computationally demanding. In order to select the most representative source zone among the thousands of possible offshore earthquake ruptures, unsupervised machine learning is needed. Using this type of AI will more accurately determine representative earthquake ruptures from offshore tsunami wave heights than previous interpretive methods that are subject to significant uncertainty. The result will yield a transparent and consistent methodology for tsunami source selection.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Acquisition and/or Development,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Mendenhall postdoctoral fellow using machine learning,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"The Global Marine Minerals Project, with the Coastal and Marine Hazards and Resources Program, currently employs a Mendenhall postdoctoral fellow who is using a machine learning approach to predict the composition of seafloor massive sulfide deposits. The fellow proposed this work in their submission in 2023, which was accepted for funding. They are currently executing the work and preparing publicaitons.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Critical Mineral Assessment with AI Support (CriticalMAAS),Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"An AI-assisted workflow could enable the USGS to accomplish its mission, produce high-quality derivative products from raw input data, and deliver timely assessments that reduce exploration risk and support decisions affecting the management of strategic domestic resources.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Acquisition and/or Development,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Shoreline modeling,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"We're trying to evaluate the prospect of using AI/ML methods/models (e.g., LSTM, CNN, Transformers) to predict shoreline evolution and compare their accuracy to traditional physics-based models.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Cell Phone Application for Oil Spill Detection,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,We would like to develop a model that can be used to interpret cell phone images to predict oil in environmental samples. The tool can be rapidly deployed for use in the field by the oil spill responder community.,Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Acquisition and/or Development,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Wave runup and total water level observations from time series imagery at several sites with varying,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,Tool is used for separation (segmentation) of land and water in images. The resulting mask is used to calculate water levels. Tool will be used to compare to forecasted water levels and may be displayed on USGS webpages,Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Development of a Ploidy Distinction Application: A Machine Learning Approach for Discriminating Trip,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"A machine learning approach is being researched as a way for fisheries personnel in the field to determine the ploidy of wild-caught invasive carp. The use of machine learning in biomedical imaging accumulates substantial volumes of data via digital images, applies computational tools for processing these data, and can combine images from various microscopes by enhancing and equalizing image resolution. Thus, a computational model was trained with selected brightfield microscopic images of whole blood smears made from known diploid and triploid Grass Carp. Digital images were generated by using two types of microscopes: (1) a laboratory-dedicated, high caliber scope, and (2) a portable microscope meant for use in the field.",see description,Initiated,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Storm Induced Erosion Response Network,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,Provide the next generation of Total Water Level and Coastal Change Forecast,see description,Initiated,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Coastal Ecosystem Prediction System,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"The USGS is partnering with NOAA National Centers for Coastal Ocean Science to co-develop an integrated, national-scale framework for projecting wetland vulnerability to sea level rise. Multi-model ensemble predictions will show a range of possible future conditions and inform the protection of coastal communities, economies and their natural resources.",see description,Initiated,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, National Wildlife Disease Database,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"The National Wildlife Health Center (NWHC) has contracted the Pacific Northwest National Laboratory (PNNL) to build a national wildlife disease database (NWDD). Funded through the American Rescue Plan Act of 2021 (ARPA), the NWDD will bring together various wildlife health data streams across informational domains (i.e., laboratory results, environmental observations, news media, etc.) to provide situational awareness and advanced analytics to natural resource authorities around the country. As part of the NWDD, PNNL plans to integrate their Canvas software. Canvas is a machine learning, AI, and data science tool that can visualize and contextualize information from one or more sources.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Acquisition and/or Development,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Sediment Transport in Coastal Environments (funded by San Francisco Bay-Delta PES),Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,Machine learning based time-series imputation of oceanographic time-series,Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, "Machine learning based shoreline time-series imputation, classification and forecasting (time-series",Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"Supports basic data generation and QC/QA procedures, for large scale and short-term forecasting of shoreline trends",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, National Oceanographic Partnership Program (NOPP),Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,Machine Learning based coastal sediments assessment and prediction,Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, RSCC and TCA projects.,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"Machine Learning (ML) methods for identifying, assessing, and quantifying coastal features and habitats and coastal change hazards.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Machinine Learning based shoreline detection and sea ice dynamics using coastal cameras,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"Supports basic data generation and QC/QA procedures, for large scale and short-term forecasting of shoreline trends and sea-ice dynamics in coastal environments",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Acquisition and/or Development,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Using Machine Learning in USGS StreamStats to make suspended sediment and bedload predictions,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,A tool for resource managers who need estimates of suspended sediment and bedload in Minnesota rivers without sampling data.,Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Operation and Maintenance,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Merbok Supplemental,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"Machine Learning based shoreline detection and mapping, automated data suitability analyses from satellite imagery",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Annual National Land Cover Database Deep Learning,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,Using geospatial artificial intelligence methods to map annual land cover and other land surface characteristics for the Nation.,Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Operation and Maintenance,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, "Event and sequence (life-stage, behavior, activity, movement modality) identification, segmentation,",Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"Animal movement data reflect various activities from individuals, including the use and availability of habitats and resources necessary for survival, behaviors such as feeding, flying, or nesting, and interactions with neighbors, predators, and prey. These biological patterns and processes are not readily observable from remotely tracked individuals and require identification from complex data streams. Complete representation of desired activites and behaviors may require multiple individual tools to satisfy all objectives.",see description,Initiated,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Data-driven approaches to filling missing time-series data within the San Francisco Bay-Delta,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"In measurements of natural systems, incomplete time-series data are the rule, not the exception. Environmental time-series data may suffer from gaps at a variety of time scales, significantly reducing the number of observations to understand phenomena, identify change, calibrate models, and predict future behavior. Data may be missing because of sensor degradation (e.g., biofouling, mechanical issues, power failure), failure to meet basic quality assurance checks, or resampling of paired observations to a common timestamp. Decades of water-quality time-series data (e.g., turbidity, salinity, temperature) collected throughout the San Francisco Bay-Delta (SFBD) by a variety of agencies including USGS, CA DWR, USBR, USFWS, and USACE are used as a critical indicator of estuary health. These data are no exception to the rule and contain numerous gaps. While bad data points can be relatively straightforward to identify (e.g., as anomalously high or low values), they are challenging to replace and are often flagged and left blank, which can bias long-term observational records. We will test and develop several (~5) proposed methods to fill gaps in water-quality time-series data and quantify the uncertainty in those estimates. The methods will range in complexity, from linear regression to machine learning and deep learning approaches (Lepot and others, 2017), and will characterize uncertainty in the filled data (Cox and others, 2003). For this proposal, we will initially restrict our scope to turbidity data, because of the relatively high biofouling rate of the optical sensors typically used to measure turbidity, as well as its ecological significance. In addition, we are already actively implementing one of the methods to fill gaps in turbidity data collected in a south S.F. Bay salt-marsh tidal creek (Figure 1) as part of a project investigating temporal variability in sediment delivery to marshes funded by the S.F. Bay Regional Monitoring Program.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Acquisition and/or Development,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Seabird and Marine Mammal Surveys Near Potential Renewable Energy Sites Offshore Central and Souther,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.," Using rapidly developing machine- learning (ML) techniques, the USGS WERC team is developing new methods to automate the detection and counts of seabirds and marine mammals from digital imagery.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Data-driven approaches to filling missing time-series data within the San Francisco Bay-Delta,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"In measurements of natural systems, incomplete time-series data are the rule, not the exception. Environmental time-series data may suffer from gaps at a variety of time scales, significantly reducing the number of observations to understand phenomena, identify change, calibrate models, and predict future behavior. Data may be missing because of sensor degradation (e.g., biofouling, mechanical issues, power failure), failure to meet basic quality assurance checks, or resampling of paired observations to a common timestamp. Decades of water-quality time-series data (e.g., turbidity, salinity, temperature) collected throughout the San Francisco Bay-Delta (SFBD) by a variety of agencies including USGS, CA DWR, USBR, USFWS, and USACE are used as a critical indicator of estuary health. These data are no exception to the rule and contain numerous gaps. While bad data points can be relatively straightforward to identify (e.g., as anomalously high or low values), they are challenging to replace and are often flagged and left blank, which can bias long-term observational records. We will test and develop several (~5) proposed methods to fill gaps in water-quality time-series data and quantify the uncertainty in those estimates. The methods will range in complexity, from linear regression to machine learning and deep learning approaches (Lepot and others, 2017), and will characterize uncertainty in the filled data (Cox and others, 2003). For this proposal, we will initially restrict our scope to turbidity data, because of the relatively high biofouling rate of the optical sensors typically used to measure turbidity, as well as its ecological significance. In addition, we are already actively implementing one of the methods to fill gaps in turbidity data collected in a south S.F. Bay salt-marsh tidal creek (Figure 1) as part of a project investigating temporal variability in sediment delivery to marshes funded by the S.F. Bay Regional Monitoring Program.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Acquisition and/or Development,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Image classification from images on robotic platforms,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"Tool identifies different terrains based on training on labeled images. Application developed for NASA Mars rovers, but applicable elsewhere.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, ChatGPT to write Python scripts for ArcGIS Pro Maps to be CVD-Friendly,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.," I do not know how to code, but we are attempting to automate the process of changing colors of planetary geologic map units within ArcGIS pro so that they are all color-vision deficiency friendly. This process is very time-consuming when done by hand, so we're using ChatGPT to help write scripts about this.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Acquisition and/or Development,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Analytical Chemistry & Geologic Resources Identification from Laser Spectroscopy,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"Determining composition of geologic, mineral, and resource targets from laser-based spectroscopic platforms",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Analytical Chemistry & Geologic Resources Identification from X-rays,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above., Determining chemistry and rock composition from X-ray-based analytical chemistry platforms,Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Retrieval-Augmented Generation Using USGS Documentation,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,Generative AI chatbots continue to make inroads across our lives. This project seeks to test creating and using a RAG model on USGS documentation with the ultimate goal of creating an internal only chat bot that has knowledge of internal USGS documentation.,see description,Initiated,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Foundation Models to Advance Earth Science,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"Advance the understanding of Earth's conditions and processes by developing and deploying generalist AI models (Foundation Models) trained on Earth Observations from field, suborbital, and orbital sensors. These models, along with the resulting insights, will empower scientists and land user managers to achieve more while also advancing the broader field of AI and machine learning science.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Rangeland Condition Monitoring Assessment and Projection (RCMAP),Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"Rangelands, comprised of open grasslands and/or shrublands, occupy huge swathes of land in the US, typically where climate and/or soils are too harsh for either forest or agriculture. Rangeland ecosystems provide critical wildlife habitat (e.g., greater sage grouse, pronghorn, black-footed ferret), forage for livestock, carbon sequestration, provision of water resources, and recreational opportunities. At the same time, rangelands are vulnerable to climate change, fire, and anthropogenic disturbances. The arid-semiarid climate in most rangelands fluctuates widely, impacting livestock forage availability, wildlife habitat, and water resources. Many of these changes can be subtle or evolve over long time periods, responding to climate, anthropogenic, and disturbance driving forces. To address the need for long-term tracking of vegetation change, scientists from the USGS and Bureau of Land Management (BLM) developed the Rangeland Condition Monitoring Assessment and Projection (RCMAP) project. RCMAP provides maps of vegetation cover at yearly time-steps, a critical refence to advancing science in the BLM and assessing Landscape Health standards. RCMAP quantifies the percent cover of ten rangeland components (annual herbaceous, bare ground, herbaceous, litter, non-sagebrush shrub, perennial herbaceous, sagebrush, shrub, and tree cover and shrub height) at yearly time-steps across the western U.S. using field training data, Landsat imagery, and machine learning.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Operation and Maintenance,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Global Food Security-Support Analysis Data (GFSAD) Project,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"Climate variability and ballooning populations are putting unprecedented pressure on agricultural croplands and their water use, which are vital for ensuring global food and water security in the twenty-first century. In addition, the COVID-19 pandemic, military conflicts, and changing diets have added to looming global food insecurity. Therefore, there is a critical need to produce consistent and accurate global cropland products at fine spatial resolution (e.g., farm-scale, 30m or better), which are generated consistently, accurately, and routinely (e.g., every year). Therefore, the overarching goal of this continuity NASA MEaSUREs (Making Earth System Data Records for Use in Research Environments) proposal is to develop a comprehensive global food security-support analysis data (GFSAD) project that will produce multiple cropland models, maps, and monitoring tools leading to a wide array of products using machine learning algorithms (MLAs), Artificial Intelligence (AI), and satellite sensor big-data analytics through cloud-computing. The GFSAD project, will produce four distinct Landsat-derived global cropland products. These products are: 1. Global Cropland Extent Product @ 30m (LGCEP30-2020, LGCEP30-2025). 2. Global Rainfed and Irrigated Product @ 30m (LGRIP30-2020, LGRIP30-2025). 3. Global Cropping Intensity Product @ 30m (LGCIP30-2020 & LGRIP30-2025) & 4. Global Crop Type Product @ 30m for USA, Canada, and India (LGCTY30-2020USACAN, LGCTY30-2025USACAN; LGCTY30-2020India, LGCTY30-2025India). The GFSAD Data is released through NASA's LP DAAC (links below): https://lpdaac.usgs.gov/news/release-of-gfsad-30-meter-cropland-extent-products/ https://lpdaac.usgs.gov/news/release-of-lgrip30-data-product/ https://lpdaac.usgs.gov/products/gfsad1kcdv001/ https://lpdaac.usgs.gov/products/gfsad1kcmv001/ GFSAD Data can be visualized at: https://www.usgs.gov/apps/croplands/app/map GFSAD website for further information: www.usgs.gov/wgsc/gfsad30",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Acquisition and/or Development,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Extracting analysis ready information from narrative geologic descriptions,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"I work in the geologic mapping group in the USGS on a new national synthesis of geologic maps. The different rocks depicted on geologic maps are typically described in narratives of 1-2 paragraphs, but often contain sparse details users would like to be able to search for/operate on. For example, how thick a unit is, analytical measurements associated with that unit, the presence of certain of-interest materials. From some experimenting with the web interface it seems the doi-hosted chatgpt does a great job extracting this sort of information with relatively simple prompts. We were hoping to use this programmatically to more systematically parse information in our geologic map synthesis database. At present, this is ~10k descriptions that consist of on average ~250 characters of text, and I would seek to run a few prompt versions on each unit (plus some testing) - so perhaps a few 10s of thousands of requests. This would then be evaluated for accuracy against a subset of human-derived extractions.",see description,Initiated,Neither,,,,,Developed in-house.,,No,,,Yes,USGS Fundamental Science Practicies Policies,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Science Support System (SSS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, 21st Century Prospecting: AI-assisted Surveying of Critical Mineral Potential,Department of the Interior,DOI,USGS,Mission-Enabling,,None of the above.,"Based on the Congressional mandate to assess critical minerals distributions in the US, the USGS Mineral Resources Program has partnered with DARPA. The objective of this partnership is to accelerate advances in science for understanding critical minerals, assessing unknown resources, and increase mineral security for the Nation. As part of an extensive suite of tools, large language models are being used to extract information from text in scanned paper geologic maps as well as mining reports (mostly government documents). (This is a re-submission as part of the new use case process)",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Acquisition and/or Development,Neither,,,,,Data not reported by submitter and will be updated once additional information is collected,Data not reported by submitter and will be updated once additional information is collected,No,,,,,,No,Yes,,Data not reported by submitter and will be updated once additional information is collected,Data not reported by submitter and will be updated once additional information is collected,,,,,,,,,,,,,,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Articulate Training Develpment AI Add On,Department of the Interior,DOI,OS - Office of the Secretary of the Interior,Mission-Enabling,,None of the above.,The Office of Emergency Management's Interior Operations Center uses Articulate to create online training courses for OEM staff. Articulate has introduced an AI add on that will expedite course creation and editing. The AI is a realistic text to speech solution but in the future will also assist with course content creation.,create online training courses for OEM staff.,Initiated,Neither,,,,,,,No,,,,,,,,,,,,,,,,,,,,,,,,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Machine Learning Applied to Geotechnical Engineering: Statistical Methods Applied to Seismic Analysis 1,Department of the Interior,DOI,BOR,Mission-Enabling,,None of the above.,Purpose is to create machine learning models that predict soil liquefaction during a seismic event and compare to the geotechnical case history database to gain confidence in the machine learning estimates. The expected benefits are the existence of another tool for geotechnical staff to predict the probability of liquefaction and reducing the amount of more complex approaches to be conducted.,"Prediction of liquefaction potential at input site, as it compares to existing case history dataset. ",Implementation and Assessment,Neither,10/1/2018,6/1/2021,1/1/2023,,Developed in-house.,,No,,,,"Activity as proof of concept and did not yield influential scientific information, so IQA not applicable.",No,No,Yes,,"Non-sensitive, mission data describing resource conditions relevant to the condition assessment/prediction question being addressed via AI-ML.",,,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Machine Learning Applied to Geotechnical Engineering: Statistical Methods Applied to Seismic Analysis 1,Department of the Interior,DOI,BOR,Mission-Enabling,,None of the above.,"Purpose is to create machine learning models to predict crest movement of an embankment dam during a seismic event. The expected benefits are the creation of a screening-level tool to be used prior to large investments in field exploration, lab testing, and large scale modeling efforts.","screening-level prediction of deformation of embankments, dams, or other facilities due to seismic loading. ",Acquisition and/or Development,Neither,12/19/2016,,,,Developed in-house.,,No,,,,"Actitivity as proof of concept and did not yield influential scientific information, so IQA not applicable.",No,No,Yes,,"Non-sensitive, mission data describing resource conditions relevant to the condition assessment/prediction question being addressed via AI-ML.",,,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Machine Learning Working Group,Department of the Interior,DOI,BOR,Mission-Enabling,,None of the above.,"Description: Perform networking and reach out to Reclamation employees to gauge staff experience and interest in creating machine learning models. Also document potential applications of machine learning to various problems encountered by staff at Reclamation in their daily work. How it helps: This gives us a better idea of the in-house capabilities of staff at Reclamation and of the potential uses that machine learning can play to help solve problems faced by Reclamation. Stage of development: Ongoing discussion and information gathering with staff is occurring. ",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Acquisition and/or Development,Neither,8/1/2020,,,,Developed in-house.,,No,,,,"Actitivity as proof of concept and did not yield influential scientific information, so IQA not applicable.",No,No,Yes,,"Non-sensitive, mission data describing resource conditions relevant to the condition assessment/prediction question being addressed via AI-ML.",,,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Machine Learning for Chemical Savings at Reverse Osmosis Plants,Department of the Interior,DOI,BOR,Mission-Enabling,,None of the above.,Chemical use for cleaning membranes in reverse osmosis (RO) water treatment facilities can contribute significant costs to the treatment process. Reclamation’s Desalination and Water Purification Research Program (DWPR) funded GHD to apply machine learning techniques to optimize Clean In Place chemical usage at RO plants. The techniques analyze water chemistry data to inform application of chemicals and result in less chemical per unit of water produced without sacrificing operational uptime or membrane life. The project was implemented at full scale operational facilities of the Water Replenishment District located in Southern California. The project demonstrated that machine learning can be a useful tool to optimize cleaning frequency of RO ystems.,Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,4/14/2023,,,,Developed with contracting resources.,Data not reported by submitter and will be updated once additional information is collected,No,,,,"Actitivity as proof of concept and did not yield influential scientific information, so IQA not applicable.",No,No,Yes,,"Non-sensitive, mission data describing resource conditions relevant to the condition assessment/prediction question being addressed via AI-ML.",Data not reported by submitter and will be updated once additional information is collected,,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Machine Learning Refines Quagga Habitat Suitability,Department of the Interior,DOI,BOR,Mission-Enabling,,None of the above.,"Extensive research has been conducted to prevent the spread of quagga mussels, a costly and damaging invasive species. Notably, several detected introductions failed to establish populations in hypothetically suitable waterbodies by pH and calcium levels. To better parameterize quagga habitat suitability, we collected ecological data at 20 stations across four invaded and two uninvaded, connected waterbodies in Arizona during 2021-2023. Data were analyzed by gradient boosted machine, an ensemble machine learning algorithm that aggregates iteratively optimized decision trees. Results identified water conditions and plankton taxa that further differentiated quagga-invaded from uninvaded stations, within a system of pH- and calcium-suitable waterbodies.",Outputs were not documented and DOI will update the use cases once additional data is collected from submitter,Implementation and Assessment,Neither,1/1/2020,,1/1/2024,,Developed in-house.,,No,,,,"Actitivity as proof of concept and did not yield influential scientific information, so IQA not applicable.",No,No,Yes,,"Non-sensitive, mission data describing resource conditions relevant to the condition assessment/prediction question being addressed via AI-ML.",,,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Generative AI to Improve Visitor Experience on NPS.gov,Department of the Interior,DOI,NPS,Mission-Enabling,,None of the above.,This is a proof of concept to us generative AI to extract data from NPS.gov and the NPS API to bring forth relevant content to visitors based on topics of interest allowing for improved trip planning. This proof of concept enriches structured data without requiring parks to create new content and reduced the time and labor cost of re-creating content. ,,Initiated,Neither,,,,,,,,,,,"Actitivity as proof of concept and did not yield influential scientific information, so IQA not applicable.",,,,,,,,,,,,,,,,,,,,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Autonomous Drone Inspections,Department of the Interior,DOI,BSEE,Science & Space,,None of the above.,"BSEE has entered into an Interagency Agreement with the Massachusetts Institute of Technology (MIT) Lincoln Laboratory (LL) to research autonomous unmanned aerial systems (UAS) and sensor technologies, with the goal of extending BSEE’s inspection capabilities to previously inaccessible environments on the Outer Continental Shelf (OSC). This research aims to build upon and leverage the work conducted by NASA's Advanced Supercomputer Division on corrosion models developed for the Level 1 Survey Report's Corrosion Level Classifications.","Multiple outputs for inspections, including decisions on corrosion levels, if a platform is boardable, does it have methane leaks.",Acquisition and/or Development,Neither,5/13/2022,10/1/2024,,,Developed with contracting resources.,IAA E24PG00019,No,,,No,"IQA not applicable. Project is for research and proof of concept, not in production.",No,No,Other,Most of the data needed is cataloged but some data points are not.,The model development stage has not yet begun. Currently in information gathering stage.,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",N/A,,,,N/A,No,,Less than 6 months,Other,Project is being developed outside BSEE's IT infastructure.,Other,Project is being developed outside BSEE's IT infastructure.,Other,Project is being developed outside BSEE's IT infastructure.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Well Activity Report Classification,Department of the Interior,DOI,BSEE,Science & Space,,None of the above.,Researching the use of Masked Language Models and Convolutional deep neural networks to identify classification systems for significant well event using data from Well Activity Reports. ,Decision on what type of siginificant event a well activity report should report.,Acquisition and/or Development,Neither,8/1/2022,8/1/2022,,,Developed in-house.,,No,,,No,"IQA not applicable. Project is for research and proof of concept, not in production.",No,No,Other,Most of the data needed is cataloged but some data points are not.,Data in oracle tables and unstructured attachment data.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",N/A,,,,N/A,No,,6-12 months,No,,No,,Other,Getting access to most tools has been timely but budget constraints make it hard to acquire resources for larger complex AI models that need more computational power.,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Level 1 Survey Report Corrosion Level Classification,Department of the Interior,DOI,BSEE,Science & Space,,None of the above.,"Offshore operators conduct Level 1 surveys annually to report on platform structural integrity, as mandated by 30 CFR 250.901(a)(7), and submit these surveys to BSEE. Each survey includes a corrosion assessment of the platform with accompanying photos. Each area is assigned a coating grade and are key indicators of a platforms overall structural health. Currently, BSEE manually reviews each report to determine if a platform requires further audits, a process that is both time and labor intensive. To expedite this, BSEE has been collaborating with NASA's Advanced Supercomputer Division to develop a machine learning model designed to flag potential mislabeling of corrosion within the survey reports. The current version of the model has shown promising accuracy in identifying sections with potential misratings, especially in areas with the most severe corrosion. BSEE and NASA will continue to focus on further improving the model's accuracy while also exploring ways to integrate it into BSEE's IT infrastructure and operational workflows for potential future implementation.",Corrosion Level Decision,Acquisition and/or Development,Neither,10/1/2018,3/24/2021,,,Developed with contracting resources.,IAA E21PG00001,No,,,No,"IQA not applicable. Project is for research and proof of concept, not in production.",No,No,Other,Most of the data needed is cataloged but some data points are not.,Unstructured Level 1 Surveys,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",N/A,,,,N/A,No,,Less than 6 months,Other,Project is being developed outside BSEE's IT infastructure.,Other,Project is being developed outside BSEE's IT infastructure.,Other,Project is being developed outside BSEE's IT infastructure.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Well Risk Assessment,Department of the Interior,DOI,BSEE,Science & Space,,None of the above.,"NASA's Advanced Supercomputer Division will utilize the work performed in the sustained casing pressure research to explore the development of machine learning models to identify various precursors of risk factors for wells. By identifying these risk factors it would help inform BSEE engineers of potential problems with the well during its various stages of development. Currently, BSEE and NASA are focusing on identifying the precursors to sustained casing pressure and evaluating their contribution to the overall risk of a well.",Well risk score,Acquisition and/or Development,Neither,3/21/2024,3/21/2024,,,Developed with both contracting and in-house resources.,IAA E21PG00001,No,,,No,"IQA not applicable. Project is for research and proof of concept, not in production.",No,No,Other,Most of the data needed is cataloged but some data points are not.,Data in oracle tables and unstructured attachment data.,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",N/A,,,,N/A,No,,Less than 6 months,Other,Project is being developed outside BSEE's IT infastructure.,Other,Project is being developed outside BSEE's IT infastructure.,Other,Project is being developed outside BSEE's IT infastructure.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Sustained Casing Pressure Identification,Department of the Interior,DOI,BSEE,Science & Space,,None of the above.,"Well casing pressure requests are submitted to BSEE to determine whether a well platform is experiencing a sustained casing pressure (SCP) problem. SCP is usually caused by gas migration from a high-pressured subsurface formation through the leaking cement sheath in one of the wells casing annuli, but SCP can also be caused by defects in tube connections, downhole accessories, or seals. Because SCP can lead to major safety issues, quickly identifying wells with SCP could greatly mitigate accidents on the well platforms. BSEE entered into an Inter-Agency Agreement with NASA's Advanced Supercomputing Division to help research the use of various AI techniques.",Sustained Casing Pressure Prediction,Acquisition and/or Development,Neither,10/1/2018,3/24/2021,,,Developed with contracting resources.,IAA E21PG00001,No,,,No,"IQA not applicable. Project is for research and proof of concept, not in production.",No,No,Other,Most of the data needed is cataloged but some data points are not.,Currently no agency owned data is being used to train and test the model.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",N/A,,,,N/A,No,,Less than 6 months,Other,Project is being developed outside BSEE's IT infastructure.,Other,Project is being developed outside BSEE's IT infastructure.,Other,Project is being developed outside BSEE's IT infastructure.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, ROV Smart Touch Subsea Pipeline Inspections,Department of the Interior,DOI,BSEE,Science & Space,,None of the above.,"The BSEE funded project with the University of Houston is researching the development a state-of-the-art robotic system, ""Smart Touch,"" to enhance subsea pipeline inspections by integrating advanced robotics and machine learning technologies. The system uses Remote Operated Vehicles (ROVs) equipped with stress wave-based smart touch sensors, video cameras, and scanning sonars to detect flange loosening and pipeline leaks autonomously. By leveraging machine learning for active sensing and robotic navigation, the project aims to improve inspection efficiency, reduce reliance on human operators, and mitigate risks associated with subsea operations. Collaborations with industry will support development, testing, and potential commercialization.",Bolt and flange tightness level prediction,Acquisition and/or Development,Neither,7/27/2023,7/27/2023,,,Developed with contracting resources.,Firm Fixed Price 140E0123C0007,No,,,No,"IQA not applicable. Project is for research and proof of concept, not in production.",No,No,Other,Most of the data needed is cataloged but some data points are not.,Data in oracle tables and unstructured attachment data.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",N/A,,,,N/A,No,,Less than 6 months,Other,Project is being developed outside BSEE's IT infastructure.,Other,Project is being developed outside BSEE's IT infastructure.,Other,Project is being developed outside BSEE's IT infastructure.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Form Recognition model for Benefits Forms,Department of Labor,DOL,OWCP,Government Services (includes Benefits and Service Delivery),N/A,None of the above.,Custom machine learning model to extract data from complex forms to tag data entries to field headers. The input is a document or scanned image of the form and the output is a JSON response with key/value pairs extracted by running the form against the custom trained form recognition model.,The AI system output is a recommendation.,Operation and Maintenance,Neither,10/1/2022,10/1/2022,3/1/2023,,Developed with both contracting and in-house resources.,"1605DC19A0004 NNG15SD60B",No,,,No,N/A,Yes,Yes,No,N/A,"No specific dataset. The AI engine came ready off the shelf, and the final product has been tested ad hoc by agency users and their NLP free text data. ","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",N/A,N/A,No,No – agency does not have access to source code.,N/A,Yes,Azure,Less than 6 months,Yes,N/A,Yes,N/A,Yes,N/A,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Language Translation,Department of Labor,DOL,Department Wide Initiative,Mission-Enabling (internal agency support),N/A,None of the above.,"To automatically translate unofficial documents into various languages quickly and easily, without the time and expense needed for human translation. ",Language translation of published documents and websites using natural language processing models.,Initiated,Neither,9/1/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Audio Transcription,Department of Labor,DOL,OWCP||VETS||WHD,Mission-Enabling (internal agency support),N/A,None of the above.,Transcription of speech to text for records keeping using natural language processing models.,Text transcription,Operation and Maintenance,Neither,3/22/2023,3/22/2023,3/22/2023,,Developed with contracting resources., NNG15SD60B,No,N/A,N/A,No,N/A,No,No,Other,Agency datasets are not necessary for the development of this use case.,N/A,Documentation is complete,N/A,N/A,No,No – agency does not have access to source code.,N/A,Yes,Azure,Less than 6 months,Yes,N/A,N/A,N/A,Yes,N/A,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Yes,,,,,,,,,,,,,,,,,, Text to Speech Conversion,Department of Labor,DOL,EBSA||OHR,Mission-Enabling (internal agency support),N/A,None of the above.,Text to speech (Neural) for more realistic human sounding applications using natural language processing models.," Audio files used on Power point presentations,",Operation and Maintenance,Neither,1/22/2023,1/22/2023,1/22/2023,,Developed with contracting resources.,NNG15SD60B,No,N/A,N/A,No,N/A,No,NO,Other,Agency datasets are not necessary for the development of this use case.,N/A,Documentation is complete,N/A,N/A,No,No – agency does not have access to source code.,N/A,Yes,Azure,Less than 6 months,Yes,N/A,N/A,N/A,Yes,N/A,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Yes,,,,,,,,,,,,,,,,,, Claims Document Processing,Department of Labor,DOL,OWCP,Government Services (includes Benefits and Service Delivery),N/A,None of the above.,To identify if physician's note contains causal language by training custom natural language processing models.,Document ranking by likely relevance and sentences highlighting of potentially causal language.,Retired,"Rights-Impacting ",,,,7/16/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Website Chatbot Assistant,Department of Labor,DOL,OTAA,Mission-Enabling (internal agency support),N/A,None of the above.,"The chatbot helps the end user with basic information about the program, information on who to contact, or seeking petition case status."," More than AI, it is question and answer bot based on the pre determine question and answers, there is no AI which would give any out put",Operation and Maintenance,Neither,2/22/2023,4/23/2023,5/10/2023,,Developed with both contracting and in-house resources.," NNG15SD22B 1605DC19A0004",NO,N/A,N/A,Yes,"The chatbot’s Question and Answer knowledge base was developed by expert staff members that regularly handle information requests and inquires from the public about the TAA program. These experts identified the most frequently asked questions as well as the relevant answers to these. There is a Feedback form for the website in close proximity to the chatbot window. Users can give feedback on the bot there. The chatbot also provides direct email and phone contact info for the TAA program, where users can let staff know if there are any issues or incorrect/incomplete info.",No,NO,NO,N/A,Frequently asked questions and their answers pertaining to the TAA program.,Documentation is complete,N/A,N/A,Yes,Yes,https://aws.amazon.com/lex/,Yes,AWS / DOL.gov,6-12 months,Yes,N/A,Yes,N/A,Yes,N/A,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Yes,,,,,,,,,,,,,,,,,, Data Ingestion of Payroll Forms,Department of Labor,DOL,WHD,Mission-Enabling (internal agency support),N/A,None of the above.,Custom machine learning model to extract data from complex forms to tag data entries to field headers. The input is a document or scanned image of the form and the output is a JSON response with key/value pairs," The output is a JSON response with key/value pairs extracted by running the form against the custom trained model.",Retired,Neither,8/1/2023,,,1/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Hololens,Department of Labor,DOL,OSHA,Education & Workforce,N/A,None of the above.,AI used to train Inspectors to visually inspect unsafe areas from a safe location.," Pictures of the location",Operation and Maintenance,Neither,8/21/2023,8/21/2023,8/21/2023,,Developed with both contracting and in-house resources., 1605TA20C0001,No,N/A,N/A,No,N/A,N/A,No,No,N/A,Purchased,Documentation is missing or not available,N/A,N/A,No,No – agency does not have access to source code.,N/A,No,N/A,Less than 6 months,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,,,,,,,,,,,,,,,,,, DOL Intranet Website Chatbot Assistant,Department of Labor,DOL,OSPE,Mission-Enabling (internal agency support),N/A,None of the above.," Conversational AI Assistant & DOL intranet websites to help answer common procurement questions, as well as specific contract questions.",It will answer question based on the question pertaining to the knowledge base.,Operation and Maintenance,Neither,3/1/2023,8/1/2023,10/1/2024,,Developed with both contracting and in-house resources.,"1605DC19A0004 NNG15SD60B",No,,,No,N/A,No,No,No,N/A,Agency Generated,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",N/A,N/A,No,"Yes – agency has access to source code, but it is not public.",N/A,No,Azure,Less than 6 months,Yes,N/A,Yes,N/A,Yes,N/A,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Official Document Validation,Department of Labor,DOL,OWCP,Mission-Enabling (internal agency support),N/A,None of the above.,AI detection of mismatched addresses and garbled text in official letters sent to benefits recipients.," The output is a recommendation for review of mismatched a addresses and garbled text in official letters sent to benefits recipients.",Retired,Neither,,,,2/16/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Electronic Records Management,Department of Labor,DOL,Department Wide Initiative,Mission-Enabling (internal agency support),N/A,None of the above.,"Meeting NARA metadata standards for (permanent) federal documents by using AI to identify data within the document, and also using NLP to classify and summarize documents."," The system output is decision based and summarization of documents.",Initiated,Neither,12/22/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Call Recording Analysis,Department of Labor,DOL,EBSA,Mission-Enabling (internal agency support),N/A,None of the above.," Automatic analysis of recorded calls made to Benefits Advisors in the DOL Interactive Voice Response (IVR) center. AI is not used for analysis, AI is used only for transcription"," The output is a transcription of calls. The AI is only doing the transcriptions",Operation and Maintenance,Neither,2/23/2023,4/1/2024,5/24/2024,,Developed with both contracting and in-house resources.,"1605DC19A0004 NNG15SD60B",No,N/A,N/A,No,N/A,No,NO,Other,Agency datasets are not necessary for the development of this use case.,Agency owned data to do analysis of call recordings,Documentation is complete,N/A,N/A,No,Yes,N/A,Yes,Azure,Less than 6 months,Yes,N/A,Yes,N/A,Yes,N/A,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Yes,,,,,,,,,,,,,,,,,, Automatic Document Processing,Department of Labor,DOL,OWCP,Government Services (includes Benefits and Service Delivery),N/A,None of the above.,Automatic processing of continuation of benefits form to extract pre-defined selection boxes. AI tool will extract data from the forms.," AI tool will extract data from the forms for further processing",Operation and Maintenance,Neither,2/23/2023,2/23/2023,5/29/2023,,Developed with both contracting and in-house resources.,"1605DC19A0004 NNG15SD60B",No,N/A,N/A,No,N/A,Yes,Yes,No,N/A,Agency Generated,Documentation is complete,N/A,N/A,No,No – agency does not have access to source code.,N/A,Yes,Azure,Less than 6 months,Yes,N/A,Yes,N/A,Yes,N/A,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Yes,,,,,,,,,,,,,,,,,, Automatic Data Processing Workflow with Form Recognizer ,Department of Labor,DOL,ETA,Mission-Enabling (internal agency support),N/A,None of the above.,Automatic processing of current complex workflow to extract required data. ,The output of the AI system is a consolidated data for review. The GPT will extract the data and into a spreadsheet and gets reconciled. ,Acquisition and/or Development,Neither,7/22/2023,11/30/2023,,,Developed with both contracting and in-house resources.,"NNG15SD22B, 1605DC19A0004",No,N/A,N/A,No,N/A,No,No,No,N/A,"Historical ETA-9130/ETA-9178 forms, 9178/9130 data from R1 states.",Documentation is widely available,N/A,N/A,,,,Yes,Azure,Less than 6 months,Yes,n/A,No,N/A,Yes,N/A,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Case Recording summarization,Department of Labor,DOL,OSHA,Mission-Enabling (internal agency support),N/A,None of the above.,Using an open source large language model to summarize publicly available case recording documents which are void of personal identifiable information (PII) or any other sensitive information. This is not hosted in the DOL technical environment and is reviewed by human note takers.,Summarized version of text,Retired,Neither,,,,4/23/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, OEWS Occupation Autocoder,Department of Labor,DOL,BLS,Mission-Enabling (internal agency support),N/A,None of the above.,The input is state submitted response files that include occupation title and sometimes job description of the surveyed units. The autocoder reads the job title and assigns up to two 6-digit Standard Occupational Classification (SOC) codes along with their probabilities as recommendations for human coders. Codes above a certain threshold are appended to the submitted response file and sent back to states to assist them with their SOC code assignment.,6-digit Standard Occupational Classification (SOC) codes,Operation and Maintenance,Neither,1/1/2015,1/1/2015,1/1/2018,,Developed in-house.,N/A,No,N/A,N/A,NO,N/A,No,No,Yes,N/A,Agency Generated,Documentation is complete,None,N/A,Yes,Yes,N/A,Yes,BLS Internal system,Less than 6 months,Yes,N/A,Yes,N/A,Yes,N/A,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Yes,,,,,,,,,,,,,,,,,, Scanner Data Product Classification,Department of Labor,DOL,BLS,Mission-Enabling (internal agency support),N/A,None of the above.,"BLS receives bulk data from some corporations related to the cost of goods they sell and services they provide. Consumer Price Index (CPI) staff have hand-coded a segment of the items in these data into Entry Level Item (ELI) codes. To accept and make use of these bulk data transfers at scale, BLS has begun to use machine learning to label data with ELI codes. The machine learning model takes as input word frequency counts from item descriptions. Logistic regression is then used to estimate the probability of each item being classified in each ELI category based on the word frequency categorizations. The highest probability category is selected for inclusion in the data. Any selected classifications that do not meet a certain probability threshold are flagged for human review. Benefits: real-time turnaround, productivity improvements, cost saves",Entry Level Item codes,Operation and Maintenance,Neither,1/1/2018,1/1/2018,3/1/2019,,Developed in-house.,N/A,No,N/A,N/A,NO,N/A,No,N/A,Yes,N/A,Agency Generated; respondent-provided category and product description information,Documentation is complete,None,N/A,Yes,Yes,N/A,Yes,BLS Internal system,Less than 6 months,Yes,N/A,Yes,N/A,Yes,N/A,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Yes,,,,,,,,,,,,,,,,,, Expenditure Classification Autocoder,Department of Labor,DOL,BLS,Mission-Enabling (internal agency support),N/A,None of the above.,"Purpose: assign a reported expense description from Consumer Expenditure Diary Survey respondents to expense classification categories known as item codes. Benefits: improve productivity, cost savings",BLS-internal item codes (expenditure classification categories),Operation and Maintenance,Neither,6/1/2019,1/1/2022,1/1/2024,,Developed with both contracting and in-house resources.,1605TA-21-F-00064,No,N/A,N/A,No,N/A,No,N/A,Yes,N/A,Agency-internal; expenditure descriptions and corresponding item code assignments,Documentation is complete,N/A,N/A,Yes,Yes,N/A,Yes,BLS Internal system,Less than 6 months,Yes,N/A,Yes,N/A,Yes,N/A,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Yes,,,,,,,,,,,,,,,,,, Generative AI Assistant,Department of Labor,DOL,OCIO,Mission-Enabling (internal agency support),N/A,None of the above.,"Private and secure in-house solution to evaluate business use cases that can solve problems using Generative AI models and semantic search. Example use cases include text summarization, text analysis, and document comparisons."," The AI system provides recommendations.",Operation and Maintenance,Neither,8/15/2023,8/15/2023,8/16/2024,,Developed with both contracting and in-house resources.,"1605DC19A0004 NNG15SD60B",No,N/A,N/A,No,N/A,No,Yes,Other,Agency datasets are not necessary for the development of this use case.,"No specific dataset. The AI engine came ready off the shelf, and the final product has been tested ad hoc by agency users and their NLP free text data. ",Documentation is widely available,N/A,N/A,No,No – agency does not have access to source code.,N/A,Yes,Azure Open AI and AWS,Less than 6 months,Yes,N/A,No,N/A,Yes,N/A,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Yes,,,,,,,,,,,,,,,,,, Occupation Code Suggestion For Job Duties Data,Department of Labor,DOL,ETA,Mission-Enabling (internal agency support),N/A,None of the above.,Suggest appropriate occupation code for job duties data using natural language processing and classification techniques,Automated Occupation code suggestions,Retired,Neither,,,,8/16/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Notice of Deficiency (NOD) Text Generation using Generative AI model,Department of Labor,DOL,ETA,Mission-Enabling (internal agency support),N/A,None of the above.,Application of custom Generative AI model to create Notice of Deficiency (NOD) text based on the input data,Natural language text generation,Initiated,Neither,7/17/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, PII Redaction,Department of Labor,DOL,OSHA,Mission-Enabling (internal agency support),N/A,None of the above.,Using Amazon Web Services Personal Identifying Information scrubber for ITA text fields and Named Entity Recognition to remove additional names from the text fields.,Input text returned with personal identifying information removed.,Operation and Maintenance,"Rights-Impacting ",5/20/2024,5/20/2024,11/19/2024,,Developed with both contracting and in-house resources., NNG15SD22B,NO,N/A,No,No,N/A,Yes,YES,Yes,N/A,Agency Generated,Documentation is available,Yes,,No,YES,Yes,Yes,AWS,6-12 months,Yes,N/A,Yes,N/A,Yes,N/A,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,YES,Yes,Risk of PII been published,Yes – by the CAIO,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",,"Other – Immediate human intervention is not practicable; however, an appropriate fail-safe has been implemented.",N/A - individuals are not interacting with the AI for this use case,N/A,NO,Review,General solicitations of comments from the public,N/A,,,, Website Chatbot Assistant,Department of Labor,DOL,ODEP,Mission-Enabling (internal agency support),N/A,None of the above.,The chatbot helps the end user with basic information about the Workforce Recruitment Program and information on who to contact.,Generate content from the WRP program,Operation and Maintenance,Neither,1/18/2024,3/18/2024,8/16/2024,,Developed with both contracting and in-house resources.,"NNG15SD22B 1605DC19A0004",No,N/A,N/A,Yes,YES,No,NO,NO,N/A,Agency provided list of questions and answers.,Documentation is complete,N/A,N/A,Yes,Yes,https://aws.amazon.com/lex/,Yes,AWS,Less than 6 months,Yes,N/A,Yes,N/A,Yes,N/A,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Yes,,,,,,,,,,,,,,,,,, Initial Determinations for incoming applications,Department of Labor,DOL,ETA,Mission-Enabling (internal agency support),N/A,None of the above.,AI tool to help perform initial analysis of the incoming 9141 and 9089 applications. Make initial determinations regarding whether issuance of a Request for Information is necessary. Create a standard RFI request and either (1) automatically issue the RFI before case assignment and hopefully obtain customer responses by the time the case is assigned OR (2) perform this action at case assignment and allow the assignment analyst to affirm and issue the RFI or not.,Flag cases for RFI ,Initiated,Neither,5/21/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, AI Course Design Assistant,Department of Labor,DOL,OSHA,Education & Workforce,N/A,Improving the quality of written communications using AI tools.,"AI tool to recommend the structure of a course, titles for modules, descriptions, and images. AI-powered algorithms analyze course content and quickly generate test questions and prompts for authentic assessments. (Blackboard/COTS Product, unmodified)","Generated text and images for Course outlines, descriptions, tests/assessments.",Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Worker PLUS Microsimulation Program,Department of Labor,DOL,CEO ,Government Services (includes Benefits and Service Delivery),N/A,None of the above.,"The Worker PLUS model was developed as an evolutionary iteration of the Paid Family and Medical Leave Simulator Model developed by Albelda and Clayton-Matthews (the ACM model)","The model simulates specific leave-taking behavior and outcomes (including number of leaves, leave lengths, benefit levels, and benefit eligibility) with individual workers in a state using data from the five-year American Community Survey (ACS) Public Use Microdata Sample (PUMS)",Operation and Maintenance,Neither,9/27/2017,9/27/2017,3/1/2021,,Developed with contracting resources.,DOLQ129633247,NO,N/A,N/A,Yes,"All CEO products adhere to the principles of rigor, relevance, transparency, independence, and ethics.",No,NO,Other,"All data is public, and not necessarily owned by DOL. ","CEO/WHD 2018 FMLA Employee Survey, American Community Survey",Documentation is widely available," Gender/Sex, Age, Income, Employment status",N/A,Yes,Yes,https://www.dol.gov/resource-library/microsimulation-model-worker-leave,No,N/A,More than 12 months,No,N/A,Yes,N/A,Yes,N/A,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Yes,,,,,,,,,,,,,,,,,, Computer-Assisted Coding: SOII Autocoder,Department of Labor,DOL,BLS,Mission-Enabling (internal agency support),N/A,None of the above.,"The Survey of Occupational Injuries and Illnesses (SOII) collects hundreds of thousands of narratives describing cases of work-related injury and illness annually. Using narratives and other relevant information, SOII Autocoder automatically assigns classifications for SOII elements, which include worker occupation, nature of the injury, part of body affected, event that resulted in the injury, source and secondary source (if it exists) that caused the injury. The use of SOII autocoder initially began in 2012 for review purposes, then gradually expanded to automatically assign these classification codes. In reference year (RY) 2022, 92% of all SOII elements were automatically coded, which were then subsequently validated by human staff. The SOII Autocoder is a transformer-based text classification model using millions of SOII cases as the labeled training data and a publicly available, third-party language model as a pre-trained base model. The Autocoder is trained annually in house using a GPU server that is owned and maintained by OCWC. Once trained, the Autocoder is deployed internally via REST API, autocoding batches of SOII and MSHA cases on a routine basis during the production cycle, all within the BLS network.",SOC and OIICS codes,Operation and Maintenance,Neither,1/12/2012,1/13/2013,1/15/2015,,Developed in-house.,N/A,No,N/A,N/A,NO,N/A,Yes,No,Yes,N/A,Agency Generated,Documentation is complete,None,N/A,Yes,Yes,N/A,Yes,BLS Internal System,Less than 6 months,Yes,N/A,Yes,N/A,Yes,N/A,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Yes,,,,,,,,,,,,,,,,,, CFOI Record Matching,Department of Labor,DOL,BLS,Mission-Enabling (internal agency support),N/A,None of the above.,"The Census of Fatal Occupational Injuries (CFOI) collects and publishes a complete count of work-related fatal injuries and descriptive data on their circumstances. The CFOI Record Matching programs matches records from various sources, including Occupational Safety and Health Administration (OSHA) Information System (OIS) files, news articles, and Quarterly Census of Employment and Wages (QCEW), to identify missing or inconsistent data in CFOI. The CFOI Record Matching programs use various aspects of AI in the areas of natural language processing and record matching. The CFOI-OIS matching uses random forest classifier model trained on previously matched CFOI-OIS data. The CFOI-QCEW matching uses TF-IDF vectors and cosine similarity to match establishment names in the CFOI-QCEW data. And the CFOI-CPDMS matching uses a publicly available, third-party question-answering language model to identify relevant CFOI elements from CPDMS data and then uses random forest classifier model to match CPDMS data to CFOI data. These processes are all conducted within the BLS network, and the outputs, in the form of spreadsheets, are provided once or twice a year to the staff in the CFOI program for review. These processes are all run within the BLS network.",Similarity score between records,Operation and Maintenance,Neither,9/18/2024,10/18/2024,9/20/2024,,Developed in-house.,N/A,No,N/A,N/A,NO,N/A,Yes,No,Yes,N/A,Agency Generated,Documentation is complete,Age and Sex/Gender,N/A,Yes,Yes,N/A,Yes,BLS Internal System,Less than 6 months,Yes,N/A,Yes,N/A,Yes,N/A,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Yes,,,,,,,,,,,,,,,,,, OIICS Coding,Department of Labor,DOL,OSHA,Mission-Enabling (internal agency support),N/A,None of the above.," Auto-code assigning Occupational Injury and Illness Classification (OIICS)"," Manual coding would be a time-consuming task, estimated to require around 65,500 labor hours for initial coding, plus additional time for error correction.",Initiated,Neither,11/4/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, AI Assisted coding -Microsoft GitHub Copilot,Department of Labor,DOL,OCIO,Mission-Enabling (internal agency support),N/A,None of the above.,"Automatically generate software code using Microsoft Copilot, this will reduce the time for generating code","Automatically generate software code using Microsoft Copilot, this will reduce the time for generating code",Operation and Maintenance,Neither,9/1/2024,9/1/2024,9/1/2024,,Developed in-house.,NNG15SD60B,No,,,No,N/A,No,No,No,N/A,N/A,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",N/A,N/A,No,"Yes – agency has access to source code, but it is not public.",NA,Yes,Azure,Less than 6 months,Yes,N/A,Yes,N/A,Yes,N/A,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, CPS OTC Prediction,Department of Labor,DOL,BLS,Mission-Enabling (internal agency support),N/A,None of the above.,"The BLS productivity office publishes measures of hours worked by major sector and industry. As part of the calculation for this measure, BLS uses Current Employment Statistics (CES) data on hours paid is used to estimate hours for payroll workers. Additional adjustments are made to this measure, such as removing paid time off (PTO), adding off-the-clock (OTC) hours, and adding in hours worked by self-employed and unpaid family workers. BLS uses the Current Population Survey (CPS) dataset to identify the amount of hours that are worked off-the-clock (OTC) by workers. There are some workers in the CPS dataset that do not report whether their time off was paid, or whether they get paid hourly or not. This data needs to be imputed to calculate the ratio of total hours worked to paid hours worked. A random forest model is used to predict the responses for workers that did not report this information by training it on characteristics (such as industry, occupation, education level, age, etc.) of respondents that have reported that information. The benefit of this AI use case is increased accuracy."," worker's time off was paid, or whether they get paid hourly or not",Operation and Maintenance,Neither,1/1/2022,2/1/2022,11/1/2022,,Developed in-house.,N/A,No,N/A,N/A,NO,N/A,No,No,Yes,N/A,Agency Generated,Documentation is complete,"Class of worker, education, marital status, family income, sex, age",N/A,Yes,Yes,N/A,Yes,BLS Internal system,Less than 6 months,Yes,N/A,Yes,N/A,Yes,N/A,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Yes,,,,,,,,,,,,,,,,,, Sample Refinement: Frame API,Department of Labor,DOL,BLS,Mission-Enabling (internal agency support),N/A,None of the above.,"BLS establishment-based surveys, such as the Survey of Occupational Injuries and Illnesses (SOII), the National Compensation Survey (NCS), and the Occupational Requirements Survey (ORS), use the Longitudinal Database (LDB) as the frame for their survey samples. For each of these programs, there is a significant time-lag between when the sample is drawn for a program and when data collection begins. A critical and time-consuming part of data collection is sample refinement: checking the latest Quarterly Census of Employment and Wages (QCEW) and LDB data to adjust for any changes to the sampled establishment during that time-lag. The QCEW/LDB Frame API allows fast retrieval of the latest QCEW/LDB data for a nationwide sample so that users can perform sample refinement more efficiently. The SOII program uses the Frame API to perform over 20 sample refinement comparison checks, which outputs a report for the data collectors. The Frame API uses TF-IDF vectors and cosine similarity to compare company names, mailing addresses, and unit descriptions between the survey sample and the latest QCEW/LDB data to determine what components may have changed. The Frame API also has the flexibility for the users to match records using other combinations of variables. These processes are all run within the BLS network.",Similarity score between records,Operation and Maintenance,Neither,5/20/2024,6/20/2024,9/20/2024,,Developed in-house.,N/A,No,N/A,N/A,NO,N/A,No,No,Yes,N/A,Agency Generated,Documentation is complete,None,N/A,Yes,Yes,N/A,Yes,BLS Internal System,Less than 6 months,Yes,N/A,Yes,N/A,Yes,N/A,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Yes,,,,,,,,,,,,,,,,,, CE Interview Item Code Estimation,Department of Labor,DOL,BLS,Mission-Enabling (internal agency support),N/A,None of the above.,"Custom machine learning model to predict an expense classification category (item code) for expenditures with text descriptions from Consumer Expenditure Interview Survey. Item Code Estimation assists with human function by approximating decision-making. It does not assign categories; it only suggests categories that the economist can then apply if correct, to speed up human review and decision-making. Benefits: improved productivity.",Expense Classification Category,Operation and Maintenance,Neither,10/1/2016,10/1/2016,7/19/2024,,Developed in-house.,N/A,No,N/A,N/A,NO,N/A,No,N/A,Yes,N/A,Agency-internal; expenditure descriptions and corresponding item code assignments,Documentation is complete,None,N/A,Yes,Yes,N/A,Yes,BLS Internal system,Less than 6 months,Yes,N/A,Yes,N/A,Yes,N/A,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Yes,,,,,,,,,,,,,,,,,, CE Interview Imputations,Department of Labor,DOL,BLS,Mission-Enabling (internal agency support),N/A,None of the above.,"Custom machine learning statistical models to impute missing expenditure values within the Consumer Expenditure Interview Survey when survey respondents answer ""don't know"" or refuse to provide an answer for an amount. The machine learning models are a more modern statistical method for replacing missing data than previous statistical methods. Benefits: improved statistical quality of imputations."," Imputed expense amounts",Operation and Maintenance,Neither,3/1/2020,3/1/2020,10/20/2024,,Developed in-house.,N/A,No,N/A,N/A,NO,N/A,No,N/A,Yes,N/A,Agency Generated,Documentation is complete,"age, gender, family structure, race, urbanicity, education, primary sampling unit",N/A,Yes,Yes,N/A,Yes,BLS Internal system,Less than 6 months,Yes,N/A,Yes,N/A,Yes,N/A,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Yes,,,,,,,,,,,,,,,,,, Note taking bot,Department of Labor,DOL,OCIO,Mission-Enabling (internal agency support),N/A,None of the above.," The bot will take the transcript and summarize the meeting notes",The output will be a transcript of the meeting and from the transcript summary and action items,Initiated,Neither,10/2/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Determine Occupation Codes from Text (NIOCCS),Department of Labor,DOL,OSHA,Mission-Enabling (internal agency support),N/A,Digitizing text from scanned documents or smart forms for archival purposes using AI.,"The Industry and Occupation Computerized Coding System (NIOCCS), a free tool offered by NIOSH, used to code industry and occupation from text descriptions of job titles. (Public Tool) ",Occupational codes by analyzing job descriptions.,Acquisition and/or Development,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Camtasia,Department of Labor,DOL,OCIO,Education & Workforce,N/A,Creating visually appealing presentations using AI-driven design suggestions.,Intended for enhancing screen recordings and video tutorials. Helps users created professional videos quickly.,Videos,Acquisition and/or Development,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Adobe Premier Pro,Department of Labor,DOL,OCIO,Mission-Enabling (internal agency support),N/A,Creating visually appealing presentations using AI-driven design suggestions.,"Assists video editors by automating complex tasks such as scene detection, video reframing, and color grading.--- Adobe Premiere Pro incorporates several AI components through its integration with Adobe Sensei, Adobe's artificial intelligence and machine learning platform. These AI-driven features are designed to enhance the video editing process, improve efficiency, and provide creative tools that simplify complex tasks.",Intuitive videos,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Audiate,Department of Labor,DOL,OCIO,Education & Workforce,N/A,Transcribing and summarizing a recorded meeting or interview using AI.,"Audiate primarily uses AI for Speech-to-text transcription, text-based audio and video editing, AI-generated scripts and audio, text-to-speech conversion, and multilingual translation. ",Text transcriptions from voice recordings.,Acquisition and/or Development,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, SnagIT,Department of Labor,DOL,OCIO,Education & Workforce,N/A,Digitizing text from scanned documents or smart forms for archival purposes using AI.,"SnagIT provides tools for quickly capturing, editing, and sharing visual content. SnagIt's Optional Character Recognition streamlines the process for extracting and reusing text from images.","Screenshots and screen recordings. SnagIt AI’s Grab Text Image Generation Automatic Captions Automatic Titles ",Acquisition and/or Development,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Facebook Mobile App,Department of Labor,DOL,OCIO,Mission-Enabling (internal agency support),N/A,Curating news articles and updates based on user preferences using AI.,"AI is not the primary technology of the use case and is not necessary for the use case, but rather a feature within the application that cannot be turned off. The application use case is for PR needs and external communication to the public coming from the Department. ","MetaAI offer search and chatbot assistance with the Facebook application. Ask Meta AI to search topics that matter to you, and get instant results more interactive than just text.",Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Google Authenticator Mobile App,Department of Labor,DOL,OCIO,Mission-Enabling (internal agency support),N/A,Unlocking smartphones or other devices without the need for passwords or PINs using AI-based facial recognition.,"Google Authenticator adds an extra layer of security to your online accounts by adding a second step of verification when you sign in. This means that in addition to your password, you'll also need to enter a code that is generated by the Google Authenticator app on your phone",Large number generation for matching for secure login.,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Google Maps Mobile App,Department of Labor,DOL,OCIO,Mission-Enabling (internal agency support),N/A,Planning travel routes using AI-driven map applications.,"Get there faster with real-time updates • Beat traffic with real-time ETAs and traffic conditions • Catch your bus, train, or ride-share with real-time transit info • Save time with automatic rerouting based on live traffic, incidents, and road closure • With Live View in Google Maps, see the way you need to go with arrows and directions placed right on top of your world. There’s no second guessing or missing another turn.","Recommendation systems, real-time traffic and route prediction",Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Instagram Mobile App,Department of Labor,DOL,OCIO,Mission-Enabling (internal agency support),N/A,Curating news articles and updates based on user preferences using AI.,"Keep up with friends on the fly with Stories and Notes that disappear after 24 hours. Start group chats and share unfiltered moments with your Close Friends. Share memories from recent events or trips in Feed. Turn your life into a movie and discover short, entertaining videos on Instagram with Reels. Customize your posts with exclusive templates, music, stickers and filters.","NLP, Communications",Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Microsoft Authenticator Mobile App,Department of Labor,DOL,OCIO,Mission-Enabling (internal agency support),N/A,Unlocking smartphones or other devices without the need for passwords or PINs using AI-based facial recognition.,"Use Microsoft Authenticator for easy, secure sign-ins for all your online accounts using multi-factor authentication, passwordless, or password autofill. You also have additional account management options for your Microsoft personal, work or school accounts.","Authentication, random large number generation",Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Periscope Mobile App,Department of Labor,DOL,OCIO,Mission-Enabling (internal agency support),N/A,Curating news articles and updates based on user preferences using AI.,"Periscope the place to show what’s happening in your world, creating vibrant communities and making the app what it was. The Periscope mobile app will be discontinued on March 31, 2021, but live video will live on as an important part of the conversation on Twitter.","NLP, Communications",Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, X (formerly Twitter) Mobile App,Department of Labor,DOL,OCIO,Mission-Enabling (internal agency support),N/A,Curating news articles and updates based on user preferences using AI.,"Post content for the world to see and join public conversations Stay up to date on breaking news and follow your interests Stay better informed with extra context from Community Notes Go live with Spaces for audio or stream live video Communicate privately with Direct Messages","NLP, Communications",Acquisition and/or Development,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Dragon Naturally Speaking,Department of Labor,DOL,OCIO,Mission-Enabling (internal agency support),N/A,Transcribing and summarizing a recorded meeting or interview using AI.,"Nuance Dragon Naturally Speaking is a software used for diction purposes, you can write any document, email, and use it with any applications using your voice.",NLP: text to voice. Audio output from text input,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, ZoomText and ZoomText Professional,Department of Labor,DOL,OCIO,Education & Workforce,N/A,Improving the quality of written communications using AI tools.,ZoomText Magnifier/Reader is a fully integrated magnification and reading program tailored for low-vision users.,Text to speech,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Visual Studio,Department of Labor,DOL,OCIO,Mission-Enabling (internal agency support),N/A,Collaborating in real-time using AI-assisted tools in word processors.,"Visual Studio IDE is a creative launching pad that you can use to edit, debug, and build code, and then publish an app.","Content creation. Can vary depending on the specific tasks and goals of the user. Common outputs include executable applications, libraries, or web applications. --- There are multiple version of it. Visual Studio incorporates several AI components to enhance the development experience for programmers and software engineers. These AI features are primarily aimed at improving productivity, code quality, and collaboration.",Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, ABBYY FineReader,Department of Labor,DOL,OCIO,Mission-Enabling (internal agency support),N/A,Digitizing text from scanned documents or smart forms for archival purposes using AI.,"ABBYY FineReader is an optical character recognition (OCR) system. It is used to convert scanned documents, PDF documents, and image files, including digital photos, into editable formats.","The output of ABBYY FineReader is the extracted and recognized text from the input document. It converts the text into editable formats such as Word documents, Excel spreadsheets, searchable PDFs, or plain text files. Text generation, Image to text ",Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Google Earth Pro,Department of Labor,DOL,OCIO,Mission-Enabling (internal agency support),N/A,Planning travel routes using AI-driven map applications.,"Google Earth Pro is a free desktop tool with advanced GIS and mapping features. With this tool, users can create maps, import and export GIS data, and access historical images.","A 3D representation of the Earth's surface, with satellite imagery and aerial photography providing a detailed view of different locations worldwide. --- Create and collaborate on immersive, data-driven maps from anywhere, with the new Google Earth. See the world from above with high-resolution satellite imagery, explore 3D terrain and buildings in hundreds of cities, and dive in to streets and neighborhoods with Street View's 360° perspectives. Image Generation Content creation Graphic Design Automation",Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Adobe Creative Cloud and Adobe Product Suite,Department of Labor,DOL,OCIO,Mission-Enabling (internal agency support),N/A,Creating visually appealing presentations using AI-driven design suggestions.,"After Effects - Create cinematic movie titles, intros, and transitions. Remove an object from a clip. Start a fire or make it rain. Animate a logo or character. With After Effects, the industry-standard motion graphics and visual effects software, you can take any idea and make it move. - > Audition is a comprehensive toolset that includes multitrack, waveform, and spectral display for creating, mixing, editing, and restoring audio content. This powerful audio workstation is designed to accelerate video production workflows and audio finishing — and deliver a polished mix with pristine sound. - > Bridge is a powerful creative asset manager that lets you preview, organize, edit, and publish multiple creative assets quickly and easily. - > Captivate is an authoring tool that is used for creating eLearning content such as software demonstrations, software simulations, branched scenarios, and randomized quizzes in HTML5 format. - > Character Animator uses your expressions and movements to animate characters in real time. - > Dimension - Create engaging content in 3D faster, with high-quality models, materials, and lighting. Dimension makes it easy to build brand visualizations, illustrations, product mockups, packaging designs, and other creative work. - >Dreamweaver - Adobe Dreamweaver is a proprietary web development tool from Adobe Inc. - > Fuse CC Beta is part of Creative Cloud and comes with Adobe CreativeSync, so your assets are always at your fingertips right when you need them. Save the characters you create in Fuse to your Creative Cloud Libraries and then access them directly within Photoshop to take them further. - > Illustrator - Edit, composite, and create beautiful images, graphics, and art on desktop and iPad. - > InCopy - lets copywriters and editors style text, track changes, and make simple layout modifications to a document. - > InDesign - Design and publish elegant layouts for print and digital. - > LightRoom - The cloud-based photo service - > PhotoShop - Edit, composite, and create beautiful images, graphics, and art on desktop and iPad. - > Premiere Pro is a video editing software for film, TV, the web, creative tools, and integration with other apps and services. - > XD is a vector-based user experience design tool for web apps and mobile apps.","Various types of creative content such as graphics, videos, websites, and more. --- Adobe Creative Cloud leverages AI through Adobe Sensei to enhance various applications, making creative processes more efficient, accurate, and innovative.",Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Autodesk,Department of Labor,DOL,OCIO,Mission-Enabling (internal agency support),N/A,Creating visual representations of data sets for reports and presentations using AI.,Autodesk Civil 3D® design software empowers civil engineers to meet complex infrastructure challenges in a 3D model-based environment.,"Detailed 3D models and visual simulations for civil engineering projects. --- Autodesk Civil 3D incorporates AI components and features that enhance civil engineering design and drafting processes. These features leverage machine learning and automation to improve productivity, accuracy, and efficiency in tasks related to land development, transportation, and infrastructure projects.",Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, AI Course Design Assistant,Department of Labor,DOL,OCIO,Mission-Enabling (internal agency support),N/A,Improving the quality of written communications using AI tools.,"AI tool to recommend the structure of a course, titles for modules, descriptions, and images. AI-powered algorithms analyze course content and quickly generate test questions and prompts for authentic assessments. (Blackboard/COTS Product, unmodified)","Generated text and images for Course outlines, descriptions, tests/assessments.",Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Webex Meeting Transcription,Department of Labor,DOL,OCIO,Mission-Enabling (internal agency support),N/A,Transcribing and summarizing a recorded meeting or interview using AI.,"Webex has an internal transcription feature that is used to transcribe webinars, though it is reviewed by someone before the transcript is posted.",Real-time transcriptions of spoken words during meetings.,Initiated,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Omni Page Ultimate 19,Department of Labor,DOL,OCIO,Mission-Enabling (internal agency support),N/A,Digitizing text from scanned documents or smart forms for archival purposes using AI.,Convert physical documents or image-based text into editable digital text.,"Searchable and editable text from scanned or image-based documents. Object recognition OCR Document Digitization Text Recognition",Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Tableau Public 2023,Department of Labor,DOL,OCIO,Mission-Enabling (internal agency support),N/A,Creating visually appealing presentations using AI-driven design suggestions.,"Allows users to share data visualizations, providing interactive insights.","Interactive visualizations such as charts, graphs, and dashboards that are easy to interpret and share. Data visualization Data Classification Predictive Analytics",Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Adobe Acrobat Professional Latest Version,Department of Labor,DOL,OCIO,Mission-Enabling (internal agency support),N/A,Improving the quality of written communications using AI tools.,Streamlines document workflows.,"Text-searchable PDFs, form-filled documents, and automatically structured files. Document management PDF Text Recognition (OCR) Document Layout Analysis",Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Tableau Creator (Desktop and Prep),Department of Labor,DOL,OCIO,Mission-Enabling (internal agency support),N/A,Creating visual representations of data sets for reports and presentations using AI.,"Enable analysts to connect, prepare, and visualize data.","visual data dashboards, reports, and predictive models. Data Analysis Predictive Analytics Data Classification",Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Grammarly 14,Department of Labor,DOL,OCIO,Mission-Enabling (internal agency support),N/A,Improving the quality of written communications using AI tools.,"Improve writing by correcting grammar, punctuation, and style errors while suggesting tone adjustments. NLP Grammar Correction Tone Detection","Grammatically correct, well-structured, and contextually appropriate text.",Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Adobe Creative Cloud with Acrobat Pro Latest Version,Department of Labor,DOL,OCIO,Mission-Enabling (internal agency support),N/A,Creating visual representations of data sets for reports and presentations using AI.,"Intended for enhancing creative workflows by automating repetitive tasks, improving accuracy of search functions, and enabling more advanced image manipulation and video editing. Object recognition Image Generation NLP","Images, Videos, Content",Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Microsoft Project Office 365,Department of Labor,DOL,OCIO,Mission-Enabling (internal agency support),N/A,Scheduling meetings or appointments or setting reminders using AI.,"Assists project managers in planning, scheduling, and managing resources. Project Scheduling Task automation Resource Allocation","Optimized project timelines, resource schedules, and predictive alerts.",Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Stata/MP 18,Department of Labor,DOL,OCIO,Mission-Enabling (internal agency support),N/A,Creating visual representations of data sets for reports and presentations using AI.,"Provides advanced statistical tools for data scientists and researchers to analyze large datasets and build predictive models. Statistical modeling Data Analysis Predictive Analytics","Statistical analysis reports, regression models, and predative insights based on large data sets.",Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Dragon Legal Individual (Speech Recognition) ,Department of Labor,DOL,OCIO,Mission-Enabling (internal agency support),N/A,Transcribing and summarizing a recorded meeting or interview using AI.,Helps legal professionals transcribe spoken language into text.,Outputs transcriptions of legal dictations and spoken languages.,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Speechify Text to Speech Voice Reader Chrome Extension 9,Department of Labor,DOL,OCIO,Mission-Enabling (internal agency support),N/A,Transcribing and summarizing a recorded meeting or interview using AI.,Converts written text into natural-sounding speech.,Synthesized speech from written text in various voices and languages.,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Adobe Creative Cloud / Adobe Express AI feature,Department of Labor,DOL,OCIO,Mission-Enabling (internal agency support),N/A,Creating visually appealing presentations using AI-driven design suggestions.,Helps users create visual content quickly through AI-driven automation.,"Graphics, posters, and social media content. Image Generation Content Creation Graphic Design Automation",Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Westlaw Precision & CoCounsel,Department of Labor,DOL,OCIO,Law & Justice,N/A,Searching for information using AI.,"Helps professionals find relevant case law, statutes, and legal precedents.","Legal documents, case analysis, and predictions for legal outcomes.",Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Kurzweil,Department of Labor,DOL,OCIO,Mission-Enabling (internal agency support),N/A,Transcribing and summarizing a recorded meeting or interview using AI.,"Kurzweil 1000 makes printed or electronic text accessible to people with blindness and visual impairments, speaking text aloud in a variety of natural-sounding voices that can be modified to individual preferences.",NLP: text to voice. Audio output from text input,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Microsoft Office Suite,Department of Labor,DOL,OCIO,Mission-Enabling (internal agency support),N/A,Improving the quality of written communications using AI tools.,"Microsoft Office Suite is a collection of productivity software applications developed by Microsoft. It is widely used for various personal, educational, and business tasks","create slideshows combining text, images, and multimedia. creating, editing, and formatting text documents",Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, GovChat,Environmental Protection Agency,EPA,Office of Mission Support,Government Services (includes Benefits and Service Delivery),,None of the above.,The purpose of this application is to provide a generative AI chatbot tool to EPA staff. It is materialized as a web site user interface which users interact with on their web browser when on the EPA network.,"The Use Case produces a variety of output depending on what users use it for internally which can range from content, code, and other uses of ChatGPT. A rules of behavior requires that users do not use it for final decisions or recommendations. Additionally, users will potentially use this environment for a broad set of general AI uses. Individual use cases may be described if the system becomes widely adopted and patterns of use are determined.",Operation and Maintenance,Neither,3/1/2024,3/1/2024,12/10/2024,,Developed in-house.,,No,,,No,,No,Yes,No,,"No data is used to train, etc the model.","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",Other,None,Yes,Yes – source code is publicly available.,https://github.com/usepa/msft-chatbot-ui/,Yes,EPA Azure Cloud Hosting System ,6-12 months,Yes,,Yes,,Yes,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, Transitioning historical help desk knowledge to new help desk software ,Environmental Protection Agency,EPA,Office of Air and Radiation,Energy & the Environment,,Summarizing the key points of a lengthy report using AI.,The program has 20 years of help requests and responses in a customized open source help desk tool. We are mapping out a transition to Jira Service Desk but are not able to easily import 20 years of knowledge and answers to common questions. The project team plans to use generative AI to categorize questions and summarize responses. These outputs would serve as the knowledge base for the new help desk. ,"Knowledge base to populate the new help desk system: - Common questions - Categories of questions - Responses",Initiated,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Emission monitoring program knowledge management system search and summary,Environmental Protection Agency,EPA,Office of Air and Radiation,Energy & the Environment,,Searching for information using AI.,"Provide a chatbot to respond to EPA monitoring system analyst questions about the emission monitoring programs, including historical regulatory changes and the reasons for those changes. ","Using preambles and regulatory text, respond to questions such as: - What is the grace period for performing a relative accuracy quality assurance test? - In what year were the monitoring performance criteria last updated and what was the justification? - What monitoring options are available for an oil-fired power plant?",Acquisition and/or Development,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Identification of lead service lines in drinking water distribution systems using bioindicators/biomarkers,Environmental Protection Agency,EPA,Office of Reseach and Development,Science & Space,,None of the above.,Explore the potential of predictive tools to identify lead service lines (LSLs). Alternative approach using bioindicators/biomarkers obtained from high-resolution microbiome profiles to identify LSLs without excavation.,Predict LSLs in DWDSs . Two outcomes (binary): LSL or non-LSL.,Initiated,Neither,1/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Generative AI for Study Evaluation and Document Chat,Environmental Protection Agency,EPA,Office of Research and Development,Science & Space,,None of the above.,Reduce the time required to complete study evaluation and make it easier to find/recall information found in documents.,"The outputs are generated text based upon a prompt. The text is intended to help the user answer questions about a document, specifically about the quality of the information reported.",Initiated,Neither,3/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Natural Gas STAR Videos ,Environmental Protection Agency,EPA,Office of Air and Radiation,Energy & the Environment,,None of the above.,"This is a commercial product that is not reflected in the list in the previous column. The purpose of using this AI tool was to simply create realistic audio narration files, without having to go through the lengthy and more expensive process of hiring a voice actor to read our scripts. The tool turns text into speech. ","The AI tool did not make any of the decisions or predictions. It only sounded out text that we created and supplied the tool. ",Acquisition and/or Development,Neither,11/17/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Agency Records Management System (ARMS),Environmental Protection Agency,EPA,Office of Mission Support,Mission-Enabling,,None of the above.,"purpose: to improve accuracy of Records schedule prediction expected benefit: reduce administrative burden on EPA staff ",the machine learning model predicts records schedules ,Operation and Maintenance,Neither,10/1/2020,10/1/2020,11/1/2023,,Developed in-house.,,No,,,No,N/A,Yes,Yes,No,,we used a sub-set of agency records,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",Other,n/a,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Agency Records Management System (ARMS Application),More than 12 months,Yes,,No,,No,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, Use of Microsoft CoPilot within the M365 Purview Security Suite,Environmental Protection Agency,EPA,Office of Mission Support,Mission-Enabling,,None of the above.,"Purview is one of the Agency’s tools to protect our Microsoft cloud data and endpoint devices. Several potential use cases are possible with the addition of CoPilot into the product which is anticipated in upcoming releases within our GCC tenant. Our proposed use of AI tools is limited in scope the Agency’s existing security boundary, data and Authority to Operate (ATO). The expected benefits are faster answers to common questions about the nature of EPA's security and IT infrastructure environment. We will be able to more effectively leverage the voluminous amount of data currently collected by Purview, Microsoft Security Center, and Defender agents to make decision in a more expedited fashion. We will be able to act on results more quickly to help reduce security vulnerabilities and weaknesses. ","a. Security and infrastructure analysts will be able to use CoPilot to better understand and report on existing security vulnerabilities on Microsoft Windows devices. Purview currently delivers canned reports in our dashboard which shows existing vulnerabilities. CoPilot can be queried to ask specific questions about the vulnerabilities, such as which computers are impacted and the potential mitigation strategy for specific vulnerabilities. b. CoPilot can be used to analyze and report on installed software, reporting on potential vulnerabilities and versions for each package. c. CoPilot can describe and summarize Data Loss Prevention (DLP) alerts that require Agency attention and help analysts to understand which types of sensitive data are triggering alerts and to improve our models. d. CoPilot can quickly respond to common questions about the environment without writing custom reports: how many Windows 22H2 computers are in the environment? How many systems are missing the latest Microsoft System Center Configuration Manager (SCCM) client? How many computers are still unpatched for the Log4J vulnerability? What is the last login for the workstation name: R07Smith123?",Initiated,Neither,11/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Use of AI tools within the Agency’s instance of ServiceNow,Environmental Protection Agency,EPA,Office of Mission Support,Mission-Enabling,,None of the above.,"ServiceNow is used as the Agency’s primary service desk management system for resolving incidents, problems, change controls, knowledge management, inventory of systems and software including the central Configuration Management Datbase (CMDB). Employees and IT professionals rely on this platform to obtain service, submit incidents, obtain reports and trends, and plan workloads and priorities to solve problems. Our proposed use of AI tools is limited in scope the Agency’s existing security boundary, data and ATO. The expected benefits are reduced numbers of service tickets and calls which will have a direct cost reduction. Help desk and IT professionals throughout the agency will be able to leverage the vast trove of knowledge articles, asset inventory and history of hundreds of thousands of service tickets to find answers to challenging (and common) IT issues facing our workforce.","Several use cases have emerged to leverage ServiceNow's upcoming AI tools for employees to engage with an AI agent to ask technical questions to solve specific technical issues. For example: b. An employee may ask, how do I reset my password; the answer will provide a written response with steps based on our existing internal knowledge articles and a link to the Agency’s self-service password management page at pss.epa.gov. This could divert thousands of calls away from service desk agents each year. c. A service desk agent may ask, how do I troubleshoot a problem a customer reports with the error message at Windows login, “domain not found.” The response would be a list of steps both the technician and employee will need to preform to resolve the error. This will save the technician a significant amount of time to search the ServiceNow knowledge articles, read the contents and relay instructions to the customer. d. A manager may ask the AI agent, how many computers in Region 10 require updates to the latest version of Windows 11 23H2? How many installations of Adobe Acrobat (pro) are installed in HQ? How many Dell Latitude 5340 computer have expired warranties agency-wide, provide a detailed list including service tags? ",Initiated,Neither,7/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Machine learning-assisted literature screening,Environmental Protection Agency,EPA,Office of Reseach and Development,Science & Space,,Searching for information using AI.,"The purpose is to help scientists in screening references by using machine learning to rank references by title and abstract based on relevance as determined from a set of ""on topic"" and ""off topic"" references. The benefits are that scientists will more efficiently identify relevant references for consideration in the development of scientific assessments. ","The machine learning tools support scientists in screening references by providing those references in a ranked order. That ranked order may or may not be based on seed references that are ""on topic"" and/or ""off topic. The screening tools build a model based on what the screener identifies for ""inclusion"" and ""exclusion"". This ranked set of references allows scientists to more quickly identify relevant references. Some of these tools also provide an estimate or recall, or the percentage of relevant references identified. ",Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Data Management and Analytics Platform (DMAP) Operations and Governance AI Use Cases,Environmental Protection Agency,EPA,Office of Mission Support,Mission-Enabling,,None of the above.,"Improve data quality, operational efficiency, and improved use of data to support mission outcomes","Proposed use of AI and ML to enhance DMAP mission needs and operational excellence in the following areas: Automated Data Quality Assessment • AI-augmented data profiling • Auto-detect data anomalies, missing values, outliers • Generate comprehensive data quality reports • Recommend data cleaning and transformation strategies • Provide statistical insights about dataset characteristics AI-Augmented Data Documentation Generation • AI to aid in creating human-readable data dictionaries • Generate insights into semantic relationships between data elements • AI-augmented data lineage and provenance tracking documentation AI-Driven Data Enrichment Recommendations • Suggest potential external data sources for enrichment • Recommend feature engineering techniques • Identify potential data joins and correlations • Flag opportunities for data augmentation Workflow and Pipeline Automation • AI to aid in Extract Transform Load/Extract Load Transform (ETL/ELT) pipeline configurations • Create reproducible data transformation scripts • Develop dynamic data routing and processing workflows • Implement intelligent error handling and recovery mechanisms Intelligent Onboarding Assistant • Guide users through data and project onboarding processes and workflows • Provide contextual recommendations during onboarding process • Validate/audit project requirements and compliance needs • Offer interactive tutorials and best practices Predictive Data Governance • Recommend data access controls • Detect potential compliance risks • Augment analysis/validation/auditing and recommendations for data sensitivity levels • Suggest data anonymization and masking strategies Feature Engineering for ML and Analysis • Auto-generate initial machine learning features • Identify potential model training datasets • Recommend preprocessing techniques • Highlight dataset suitability for specific ML tasks Semantic Data Discovery • Create semantic knowledge graphs • Enable natural language querying of datasets • Aid in identifying and recommending data relationships across disparate sources • Support data catalogue generation ",Initiated,Neither,10/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Intelligent Superfund Search & Chatbot capabilities,Environmental Protection Agency,EPA,Office of Land and Emergency Management,Energy & the Environment,,Searching for information using AI.,"The intended purpose of this AI use case is to modernize and enhance the EPA's Superfund Enterprise Management System (SEMS) document search and processing system through intelligent automation. Key expected benefits include: 1. Dramatic improvement in operational efficiency by reducing search time from months/weeks to minutes 2. Enhanced accuracy in document retrieval and information extraction 3. Better user experience through intuitive natural language search capabilities 4. Cost reduction through automated document processing 5. Improved compliance and quality control in environmental site management 6. Increased ability to process and derive insights from millions of technical documents 7. Better decision support for EPA staff managing contaminated sites ","The AI system will generate the following outputs: 1. Intelligent Document Processing Results: - Extracted key entities (site locations, contaminants, responsible parties) - Document classifications and metadata - Automated summaries of technical reports 2. Search and Retrieval Outputs: - Relevant document recommendations based on natural language queries - Direct answers to specific questions with source citations - Cross-document insights and connections 3. Analysis and Decision Support: - Site contamination pattern analysis - Related case recommendations - Risk assessment summaries 4. Quality Assurance Outputs: - Confidence scores for extracted information - Source attribution for all provided information ",Acquisition and/or Development,Neither,4/1/2024,4/1/2024,,,,,,,,,,,,No,,"No data is used for AI training, finetuning. Subset of agency records is used for evaluation.","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,,,,Yes,Superfund Enterprise Management System,6-12 months,No,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Resource Conservation and Recovery Act (RCRA) Predictive Analytics,Environmental Protection Agency,EPA,Office of Enforcement and Compliance Assurance,Law & Justice,,None of the above.,"Risk scoring of Large Quantity Generators (LQGs) to support RCRA inspections ","Risk scoring (integers 0 to 4, inclusive) for LQGs ",Implementation and Assessment,Safety-Impacting,7/31/2019,7/31/2019,10/12/2022,,,,,,,No,,No,No,Yes,,"From https://echo.epa.gov/tools/data-downloads : Facility Registry Service, Air Stationary Source Data Downloads, Water National Pollutant Discharge Elimination System (NPDES) Data Downloads, Water Biosolids Data Download, Hazardous Waste Data Download","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",Other,None,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Data Management and Analytics Platform (DMAP),Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Clean Water Act (CWA)Predictive Analytics,Environmental Protection Agency,EPA,Office of Enforcement and Compliance Assurance,Law & Justice,,None of the above.,"Identify high risk National Pollution Discharge Elimination System (NPDES) facilities that don't submit Discharge Monitoring Reports (DMRs) ","Risk scoring (Ranking from lowest to highest risk) for NPDES facilities that don't submit DMR's ",Initiated,Neither,7/31/2019,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Clean Air Act (CAA) Predictive Analytics,Environmental Protection Agency,EPA,Office of Enforcement and Compliance Assurance,Law & Justice,,None of the above.,"Risk scoring of Majors and Synthetic Minor Facilities to support CAA inspections ","Decimal score between 0 and 1 for Major and Synthetic Minor CAA Facilities ",Initiated,Neither,7/31/2019,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,i,,,,,,,, Extracting references from technical documents,Environmental Protection Agency,EPA,Office of the Administrator,Energy & the Environment,,Searching for information using AI.,"In response to OMB interest—which we agree with—in giving researchers more visibility when we cite their work, National Center for Environmental Economics (NCEE) is investigating various methods to document citations in economic analysis and other regulatory documents. We have explored using GovChat to extract references from regulatory documents. ","Output would be lists of unique references by document, which can then be combined into a list with counts. The list could then be used to acknowledge researchers whose work is used by EPA. ",Initiated,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Streamflow Duration Assessment Modeling,Environmental Protection Agency,EPA,Office of Water,Science & Space,,None of the above.,"Streams exhibit a diverse range of hydrologic regimes, and the hydrologic regime strongly influences the physical, chemical, and biological characteristics of active stream channels and their adjacent riparian areas. One important aspect of the hydrologic regime is streamflow duration—the length of time that a stream sustains flowing surface water. Determining streamflow duration class can help support a variety of regulatory and non-regulatory water resource management decisions. This random forest modeling supports prediction of the appropriate streamflow duration classification across a range of geographies. Full information is available at: https://www.epa.gov/streamflow-duration-assessment/random-forest-modeling","Prediction of whether a streamflow is “perennial,” “intermittent,” or “ephemeral”",Operation and Maintenance,Neither,3/6/2009,3/6/2009,11/15/2011,,,,,,,No,Each SDAM was developed under an EPA developed and approved QAPP. ,No,Yes,No,,"Pacific Northwest https://www.hydroshare.org/resource/fb4e7b8758d0478cbfe0d6c786f0f968/ Northeast and Southeast https://catalog.data.gov/dataset/nese-betasdam-final-data-and-code Arid West https://catalog.data.gov/dataset/aw-betasdam-final-data-and-code Western Mountains https://catalog.data.gov/dataset/wm-betasdam-final-data-and-code Great Plains https://catalog.data.gov/dataset/gp-betasdam-final-data-and-code ","Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",Other,None,Yes,Yes – source code is publicly available.,https://github.com/USEPA/SDAM-web-app,Yes,Data Management and Analytics Platform (DMAP) (Posit Connect),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, Solicitation Review Tool (SRT),General Services Administration,GSA,OGP,Other,Section 508 Compliance,None of the above.,"The SRT intakes SAM.gov data for all ICT solicitations. The system then compiles the data into a database to be used by machine learning algorithms. The first of these is a Natural Language Processing model that determines if a solicitation contains compliance language. If a solicitation does not have compliance language, then it is marked as non-compliant. Each agency is asked to review their data and validate the SRT predictions. GSA also conducts random manual reviews monthly.",Determination whether a solicitation is compliant or not. Text form.,Implementation and Assessment,Neither,5/1/2018,5/1/2019,6/1/2020,N/A,Developed with both contracting and in-house resources.,N/A,No,Public Experience Portfolio – GSA ,Enable increased self-service through an expansion of digital costumer service tools,No,Non-public,No,No,Yes,N/A,SAM.gov data sets,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",N/A,N/A,Yes,"Yes – agency has access to source code, but it is not public.",https://github.com/GSA/srt-fbo-scraper,Yes,SRT,6-12 months,Yes,N/A,Yes,N/A,Yes,N/A,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,No,"Benchmark evaluation: Testing of the AI model has involved the use of benchmarks (either publicly available or internally created) to estimate performance in real-world settings, but has not been tested in an operational environment. ","False negatives, and identified through random QA sessions.",Planned or in-progress,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,N/A,N/A - individuals are not interacting with the AI for this use case,N/A,No,N/A,Post-transaction customer feedback collections,N/A,"No – Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.","Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.","No – This AI use case is not subject to the opt-out requirement because the AI functionality is solely used for the prevention, detection, and investigation of fraud or cybersecurity incidents, or the conduct of a criminal investigation.",N/A Acquisition Analytics,General Services Administration,GSA,FAS,Mission-Enabling,N/A,None of the above.,Takes Detailed Data on transactions and classifies each transaction within the Government-wide Category Management Taxonomy,Acquisition Analytics,Implementation and Assessment,Neither,10/1/2018,4/1/2019,4/1/2019,NA,Developed with both contracting and in-house resources.,N/A,No,Public Experience Portfolio – GSA ,Enable increased self-service through an expansion of digital costumer service tools,No,Non-public,No,No,Yes,N/A,Metadata Link,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",N/A,N/A,Yes,"Yes – agency has access to source code, but it is not public.",It is available on the FAS Enterprise Data Architecture Github Repository,Yes,EDA,Less than 6 months,Yes,N/A,Yes,N/A,Yes,N/A,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,No,"Benchmark evaluation: Testing of the AI model has involved the use of benchmarks (either publicly available or internally created) to estimate performance in real-world settings, but has not been tested in an operational environment. ",Misclassification and subsequent misalignment of transaction throughput through GWAS categories,Planned or in-progress,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,N/A,Online - in the terms or instructions for the service.,N/A,No,N/A,None of the above,N/A,"No – Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.","Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.","No – This AI use case is not subject to the opt-out requirement because the AI functionality is solely used for the prevention, detection, and investigation of fraud or cybersecurity incidents, or the conduct of a criminal investigation.",N/A City Pairs Program Ticket Forecast and Scenario Analysis Tools,General Services Administration,GSA,FAS,Transportation,N/A,None of the above.,"Takes segment-level City Pair Program air travel purchase data and creates near-term forecasts for the current and upcoming fiscal year by month and at various levels of granularity including DOD vs Civilian, Agency, and Region.",City Pairs Program Ticket Forecast and Scenario Analysis Tools,Implementation and Assessment,Neither,10/1/2021,2/1/2022,4/1/2019,NA,Developed in-house.,N/A,No,Public Experience Portfolio – GSA ,Enable increased self-service through an expansion of digital costumer service tools,No,Non-public,No,No,Yes,N/A,Metadata Link,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",N/A,N/A,Yes,"Yes – agency has access to source code, but it is not public.",It is available on the FAS Enterprise Data Architecture Github Repository,Yes,EDA,Less than 6 months,Yes,N/A,Yes,N/A,Yes,N/A,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,No,"Benchmark evaluation: Testing of the AI model has involved the use of benchmarks (either publicly available or internally created) to estimate performance in real-world settings, but has not been tested in an operational environment. ",Over or underestimation and therefore mistargeting initiatives around KPIs ,Planned or in-progress,No monitoring protocols have been established: Necessary infrastructure needed to perform monitoring of AI systems in production is not available and/or a plan to monitor models has not yet been established.,No - Some individual decisions or actions require direct human oversight.,N/A,Online - in the terms or instructions for the service.,N/A,No,N/A,None of the above,N/A,"No – Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.","Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.","No – This AI use case is not subject to the opt-out requirement because the AI functionality is solely used for the prevention, detection, and investigation of fraud or cybersecurity incidents, or the conduct of a criminal investigation.",N/A Category Taxonomy Classifier,General Services Administration,GSA,FAS,Mission-Enabling,N/A,None of the above.,Classification of obligation transactions in to GWAS subcategories when existing FPDS coding scheme(PSC/NAICS) is insufficient for positive classification of a transaction. ,Category Taxonomy Classifier,Implementation and Assessment,Neither,6/1/2021,6/1/2021,10/1/2022,N/A,Developed in-house.,N/A,No,Public Experience Portfolio – GSA ,Enable increased self-service through an expansion of digital costumer service tools,No,Non-public,No,No,Yes,N/A,Metadata Link,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",N/A,N/A,Yes,"Yes – agency has access to source code, but it is not public.",It is available on the FAS Enterprise Data Architecture Github Repository,Yes,EDA,Less than 6 months,Yes,N/A,Yes,N/A,Yes,N/A,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,No,"Benchmark evaluation: Testing of the AI model has involved the use of benchmarks (either publicly available or internally created) to estimate performance in real-world settings, but has not been tested in an operational environment. ",Misclassification and subsequent misalignment of transaction throughput through GWAS categories,Planned or in-progress,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,N/A,N/A - individuals are not interacting with the AI for this use case,N/A,No,N/A,None of the above,N/A,"No – Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.","Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.","No – This AI use case is not subject to the opt-out requirement because the AI functionality is solely used for the prevention, detection, and investigation of fraud or cybersecurity incidents, or the conduct of a criminal investigation.",N/A CPRM Contract Breach Forecast,General Services Administration,GSA,FAS,Mission-Enabling,N/A,None of the above.,Applies obligation forecasting to ITC contracts to provide situational awareness of risk of contract ceiling breach. ,CPRM Contract Breach Forecast,Implementation and Assessment,Neither,8/1/2022,N/A,N/A,N/A,Developed in-house.,N/A,No,Public Experience Portfolio – GSA ,Enable increased self-service through an expansion of digital costumer service tools,No,Non-public,No,No,Yes,N/A,Metadata Link,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",N/A,N/A,Yes,"Yes – agency has access to source code, but it is not public.",It is available on the FAS Enterprise Data Architecture Github Repository,Yes,EDA,Less than 6 months,Yes,N/A,Yes,N/A,Yes,N/A,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,No,"Benchmark evaluation: Testing of the AI model has involved the use of benchmarks (either publicly available or internally created) to estimate performance in real-world settings, but has not been tested in an operational environment. ",1) Breach predicted too far in the future. 2) Breach predicted to early. ,Planned or in-progress,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,N/A,Online - in the terms or instructions for the service.,N/A,No,N/A,None of the above,N/A,"No – Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.","Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.","No – This AI use case is not subject to the opt-out requirement because the AI functionality is solely used for the prevention, detection, and investigation of fraud or cybersecurity incidents, or the conduct of a criminal investigation.",N/A Category Taxonomy Refinement Using NLP,General Services Administration,GSA,FAS,Mission-Enabling,N/A,None of the above.,Uses token extraction from product descriptions more accurately shape intended markets for PSCs.,N/A,Retired,Neither,N/A,N/A,N/A,11/1/2024,Developed in-house.,N/A,No,Public Experience Portfolio – GSA ,Enable increased self-service through an expansion of digital costumer service tools,No,N/A,No,No,No,N/A,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",N/A,N/A,Yes,"Yes – agency has access to source code, but it is not public.",It is available on the FAS Enterprise Data Architecture Github Repository,No,N/A,Less than 6 months,Yes,N/A,No,N/A,Yes,N/A,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,No,No testing: No testing of the model to simulate performance in an operational environment has been conducted thus far.,N/A,Planned or in-progress,No monitoring protocols have been established: Necessary infrastructure needed to perform monitoring of AI systems in production is not available and/or a plan to monitor models has not yet been established.,No - Some individual decisions or actions require direct human oversight.,N/A,N/A - individuals are not interacting with the AI for this use case,N/A,No,N/A,None of the above,N/A,"No – Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.","Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.","No – This AI use case is not subject to the opt-out requirement because the AI functionality is solely used for the prevention, detection, and investigation of fraud or cybersecurity incidents, or the conduct of a criminal investigation.",N/A Key KPI Forecasts for GWCM,General Services Administration,GSA,FAS,Mission-Enabling,N/A,None of the above.,"Takes monthly historical data for underlying components used to calculate KPIs and creates near-term forecasts for the upcoming fiscal year. Pilot effort focuses on total agency/category spend (the denominator in multiple KPIs). If the pilot program is successful, the same methodology can be extended to other KPIs.",N/A,Retired,Neither,N/A,N/A,N/A,11/1/2024,Developed with contracting resources.,N/A,No,Public Experience Portfolio – GSA ,Enable increased self-service through an expansion of digital costumer service tools,No,N/A,No,No,No,N/A,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",N/A,N/A,No,"Yes – agency has access to source code, but it is not public.",N/A,No,N/A,Less than 6 months,Yes,N/A,No,N/A,Yes,N/A,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,No,No testing: No testing of the model to simulate performance in an operational environment has been conducted thus far.,N/A,Planned or in-progress,No monitoring protocols have been established: Necessary infrastructure needed to perform monitoring of AI systems in production is not available and/or a plan to monitor models has not yet been established.,No - Some individual decisions or actions require direct human oversight.,N/A,N/A - individuals are not interacting with the AI for this use case,N/A,No,N/A,None of the above,N/A,"No – Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.","Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.","No – This AI use case is not subject to the opt-out requirement because the AI functionality is solely used for the prevention, detection, and investigation of fraud or cybersecurity incidents, or the conduct of a criminal investigation.",N/A Enterprise Brain,General Services Administration,GSA,GSA IT (IDT),Other,N/A,None of the above.,Enterprise Brain is a document repository by Tanjo (tanjo.ai) that leverages AI to improve document discovery.,N/A,Retired,Neither,10/1/2022,N/A,N/A,10/1/2023,Developed with contracting resources.,47QFCA21F0014,No,Public Experience Portfolio – GSA ,Enable increased self-service through an expansion of digital costumer service tools,No,N/A,No,No,No,N/A,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",N/A,N/A,No,No – agency does not have access to source code.,Vendor is waiting to provide source code until the project is approved to go forward based on ROI.,No,N/A,Less than 6 months,Yes,N/A,No,N/A,Yes,N/A,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,No,No testing: No testing of the model to simulate performance in an operational environment has been conducted thus far.,N/A,Planned or in-progress,No monitoring protocols have been established: Necessary infrastructure needed to perform monitoring of AI systems in production is not available and/or a plan to monitor models has not yet been established.,No - Some individual decisions or actions require direct human oversight.,N/A,N/A - individuals are not interacting with the AI for this use case,N/A,No,N/A,None of the above,N/A,"No – Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.","Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.","No – This AI use case is not subject to the opt-out requirement because the AI functionality is solely used for the prevention, detection, and investigation of fraud or cybersecurity incidents, or the conduct of a criminal investigation.",N/A AWS Captcha Solver,General Services Administration,GSA,GSA IT (IDT),Other,AWS Captcha Solver,None of the above.,"As part of a project to automate actions in AWS console using Selenium, we developed an AI model that can solve AWS's captcha so our automated system won't be blocked for being detected as a bot. The model is used in an AWS Lambda that uses a headless Selenium driver. The model was trained from scratch using Tensorflow. We have discussed making a more general form of this solution that could be applied to other AI/ML problems.",It outputs the solution to a captcha.,Initiated,Neither,1/1/2022,1/1/2022,1/1/2022,N/A,Developed in-house.,N/A,No,Public Experience Portfolio – GSA ,Enable increased self-service through an expansion of digital costumer service tools,No,Non-public,No,No,Yes,N/A,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",N/A,N/A,Yes,"Yes – agency has access to source code, but it is not public.",The source is in GSA's GitHub repository.,No,N/A,Less than 6 months,Yes,N/A,No,N/A,Yes,N/A,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,No,"Benchmark evaluation: Testing of the AI model has involved the use of benchmarks (either publicly available or internally created) to estimate performance in real-world settings, but has not been tested in an operational environment. ",None,Planned or in-progress,No monitoring protocols have been established: Necessary infrastructure needed to perform monitoring of AI systems in production is not available and/or a plan to monitor models has not yet been established.,No - Some individual decisions or actions require direct human oversight.,N/A,Other ,Was used internally for root password rotation only. No external users.,No,N/A,None of the above,N/A,"No – Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.","Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.","No – This AI use case is not subject to the opt-out requirement because the AI functionality is solely used for the prevention, detection, and investigation of fraud or cybersecurity incidents, or the conduct of a criminal investigation.",N/A ServiceNow Generic Ticket Classification,General Services Administration,GSA,GSA IT (IDT),Other,ServiceNow Generic Ticket Classification,None of the above.,"We are building a model to take generic Service Now tickets and classify them so that they can be automatically re-routed to the correct team that handles these types of tickets. The process of re-routing generic tickets is currently done manually, so the model will allow us to automate it. The initial model will target the top 5 most common ticket types.",It outputs the most likely group that a Servicenow ticket should be assigned to.,Initiated,Neither,10/1/2022,10/1/2022,N/A,N/A,Developed with contracting resources.,47QFCA21F0014,No,Public Experience Portfolio – GSA ,Enable increased self-service through an expansion of digital costumer service tools,No,Non-public,Yes,No,Yes,N/A,Service Now Ticket Data,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",N/A,N/A,Yes,No – agency does not have access to source code.,The source code is in Google Drive.,Yes,EIP,Less than 6 months,Yes,N/A,No,N/A,Yes,N/A,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,No,"Benchmark evaluation: Testing of the AI model has involved the use of benchmarks (either publicly available or internally created) to estimate performance in real-world settings, but has not been tested in an operational environment. ",None,Planned or in-progress,No monitoring protocols have been established: Necessary infrastructure needed to perform monitoring of AI systems in production is not available and/or a plan to monitor models has not yet been established.,No - Some individual decisions or actions require direct human oversight.,N/A,Other ,The team that classifies the tickets manually is aware of the AI. The end user is not notified that an AI was used.,No,N/A,None of the above,N/A,"No – Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.","Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.","No – This AI use case is not subject to the opt-out requirement because the AI functionality is solely used for the prevention, detection, and investigation of fraud or cybersecurity incidents, or the conduct of a criminal investigation.",N/A ServiceNow Virtual Agent (Curie),General Services Administration,GSA,GSA IT (IDT),Other,ServiceNow Chatbot,None of the above.,"Virtual agent that uses ML to provide predictive results for chat entries. A natural language chatbot (virtual assistant), we named Curie, as part of a multi-model customer service experience for employee's IT service requests leveraging knowledge-based articles.",Text response generated the most likely answer base on the user input. ,Initiated,Neither,N/A,N/A,N/A,N/A,Developed with contracting resources.,N/A,No,Public Experience Portfolio – GSA ,Enable increased self-service through an expansion of digital costumer service tools,No,Non-public,Yes,No,Yes,N/A,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",N/A,N/A,Yes,"Yes – agency has access to source code, but it is not public.",GSA GitHub,Yes,N/A,Less than 6 months,Yes,N/A,No,N/A,Yes,N/A,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,No,No testing: No testing of the model to simulate performance in an operational environment has been conducted thus far.,N/A,Planned or in-progress,No monitoring protocols have been established: Necessary infrastructure needed to perform monitoring of AI systems in production is not available and/or a plan to monitor models has not yet been established.,No - Some individual decisions or actions require direct human oversight.,N/A,N/A - individuals are not interacting with the AI for this use case,N/A,No,N/A,None of the above,N/A,"No – Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.","Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.","No – This AI use case is not subject to the opt-out requirement because the AI functionality is solely used for the prevention, detection, and investigation of fraud or cybersecurity incidents, or the conduct of a criminal investigation.",N/A GREX AI Document Classification,General Services Administration,GSA,PBS,Government Services (includes Benefits and Service Delivery),N/A,None of the above.,"Office of Leasing and IT Modernization partnered in a pilot program to test the feasibility of using artificial intelligence (AI) and machine learning (ML) using the AWS cloud service from FAS to classify pdf documents emailed into G-REX. The pilot built the pipeline to train individual document types that are typically uploaded in PDF format. The pilot was successful at illustrating how valuable time could be saved by completing this project in production, improving accuracy of document type assignment. The pilot utilized AWS FAS service, which would require additional funding. Appian announced they will provide the functionality as an out of the box feature in late 2024, early 2025, to be able to scan pdf document types if G-REX can move to the cloud. The cloud assessment is ongoing as of submission of the need. This was an approved FY23 Need.",N/A,Acquisition and/or Development,Neither,N/A,N/A,N/A,N/A,Developed with contracting resources.,N/A,No,Public Experience Portfolio – GSA ,Enable increased self-service through an expansion of digital costumer service tools,No,N/A,No,No,No,N/A,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",N/A,N/A,No,"Yes – agency has access to source code, but it is not public.",N/A,No,N/A,Less than 6 months,Yes,N/A,No,N/A,Yes,N/A,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,No,No testing: No testing of the model to simulate performance in an operational environment has been conducted thus far.,N/A,Planned or in-progress,No monitoring protocols have been established: Necessary infrastructure needed to perform monitoring of AI systems in production is not available and/or a plan to monitor models has not yet been established.,No - Some individual decisions or actions require direct human oversight.,N/A,N/A - individuals are not interacting with the AI for this use case,N/A,No,N/A,None of the above,N/A,"No – Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.","Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.","No – This AI use case is not subject to the opt-out requirement because the AI functionality is solely used for the prevention, detection, and investigation of fraud or cybersecurity incidents, or the conduct of a criminal investigation.",N/A Contract Acquisition Lifecycle Intelligence (CALI),General Services Administration,GSA,FAS (QP0A),Mission-Enabling,N/A,None of the above.,"CALI tool is an automated machine learning evaluation tool built to streamline the evaluation of vendor proposals against the solicitation requirements to support the Source Selection process. CALI is offered by Octo Consulting as Infrastructure as a Service and is implemented and hosted within Amazon Web Services (AWS) public cloud US East and US West regions. Once the Contracting Officer (CO) has received vendor proposals for a solicitation and is ready to perform the evaluation process, the CO will initiate evaluation by sending solicitation documents along with all associated vendor proposal documents to Bizagi’s Source Selection module, which will pass all documents to CALI. CALI will process the documents, associated metadata and begin analyzing the proposals in four key areas: format compliance, forms validation, reps and certs compliance, and requirements compliance. The designated evaluation members can review the evaluation results in CALI and submit finalized evaluation results back to Source Selection in Bizagi. CALI is currently being trained with sample data from the EULAs under the Multiple Award Schedule program.",N/A,Retired,Neither,N/A,N/A,N/A,11/1/2024,Developed with contracting resources.,N/A,No,Public Experience Portfolio – GSA ,Enable increased self-service through an expansion of digital costumer service tools,No,N/A,No,No,No,N/A,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",N/A,N/A,No,No – agency does not have access to source code.,N/A,No,N/A,Less than 6 months,Yes,N/A,No,N/A,Yes,N/A,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,No,No testing: No testing of the model to simulate performance in an operational environment has been conducted thus far.,N/A,Planned or in-progress,No monitoring protocols have been established: Necessary infrastructure needed to perform monitoring of AI systems in production is not available and/or a plan to monitor models has not yet been established.,No - Some individual decisions or actions require direct human oversight.,N/A,N/A - individuals are not interacting with the AI for this use case,N/A,No,N/A,None of the above,N/A,"No – Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.","Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.","No – This AI use case is not subject to the opt-out requirement because the AI functionality is solely used for the prevention, detection, and investigation of fraud or cybersecurity incidents, or the conduct of a criminal investigation.",N/A Survey Comment Ham / Spam Tester,General Services Administration,GSA,FAS (TTS),Other,Qualitative Data Analysis,None of the above.,The website USA.gov receives a lot of survey comments. We need a way to determine which comments are worth the time of analysts reading and which are not.,"A determination of whether the data is ""worth reading""",Retired,Neither,N/A,4/1/2020,7/1/2020,7/1/2021,Developed with both contracting and in-house resources.,N/A,No,Public Experience Portfolio – GSA ,Enable increased self-service through an expansion of digital costumer service tools,No,N/A,No,No,No,N/A,Compared to thousands of evaluations made by people,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",N/A,N/A,Yes,No – agency does not have access to source code.,"A simple custom model, but the actual rules based on the model are hidden within GCP.",No,USA.gov,Less than 6 months,Yes,N/A,No,N/A,Yes,N/A,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,No,No testing: No testing of the model to simulate performance in an operational environment has been conducted thus far.,N/A,Planned or in-progress,No monitoring protocols have been established: Necessary infrastructure needed to perform monitoring of AI systems in production is not available and/or a plan to monitor models has not yet been established.,No - Some individual decisions or actions require direct human oversight.,N/A,N/A - individuals are not interacting with the AI for this use case,N/A,No,N/A,None of the above,N/A,"No – Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.","Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.","No – This AI use case is not subject to the opt-out requirement because the AI functionality is solely used for the prevention, detection, and investigation of fraud or cybersecurity incidents, or the conduct of a criminal investigation.",N/A Classifying Qualitative Data with Medallia,General Services Administration,GSA,FAS (TTS),Other,Qualitative Data Analysis,None of the above.,"USA.gov has a lot of customer-driven qualitative data to be classified into topics: survey comments from multiple surveys, web chat transcripts, contact center agent case notes, and search box queries. It is important to know the topics of the text chunks. This is very challenging because of the massive diversity of USA.gov's content and topics covered. The focus is on Medallia, which offers natural language processing in which users can create rules based on words and their relationships with other words to tag qualitative data with our topics (passports, tax refunds, etc.). Medallia also uses AI to do classifying on its own. However, value has not been derived from this functionality and it is therefore not in use. ",Topic Groups / Classification,Acquisition and/or Development,Neither,4/1/2020,N/A,N/A,N/A,Developed with contracting resources.,N/A,No,Public Experience Portfolio – GSA ,Enable increased self-service through an expansion of digital costumer service tools,No,N/A,No,No,No,N/A,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",N/A,N/A,No,No – agency does not have access to source code.,N/A,Yes,USA.gov,Less than 6 months,Yes,N/A,No,N/A,Yes,N/A,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,No,No testing: No testing of the model to simulate performance in an operational environment has been conducted thus far.,N/A,Planned or in-progress,No monitoring protocols have been established: Necessary infrastructure needed to perform monitoring of AI systems in production is not available and/or a plan to monitor models has not yet been established.,No - Some individual decisions or actions require direct human oversight.,N/A,N/A - individuals are not interacting with the AI for this use case,N/A,No,N/A,None of the above,N/A,"No – Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.","Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.","No – This AI use case is not subject to the opt-out requirement because the AI functionality is solely used for the prevention, detection, and investigation of fraud or cybersecurity incidents, or the conduct of a criminal investigation.",N/A Elastic Machine Learning Threat Detection,General Services Administration,GSA,GSA,Other,,None of the above.,Use AI to further analyze security logs and alert on anomalous data patterns for a human SOC analyst to review. The goal is for a human/AI partnership that can more effectively and quickly search through the data.,Write anomaly data to Security Information and Event Management (SIEM) tool and create alerts for Security Operations Center (SOC) team.,Acquisition and/or Development,Neither,4/1/2022,N/A,N/A,N/A,Developed with both contracting and in-house resources.,N/A,No,Public Experience Portfolio – GSA ,Enable increased self-service through an expansion of digital costumer service tools,No,N/A,No,No,Yes,N/A,"Office of Leasing and IT Modernization partnered in a pilot program to test the feasibility of using artificial intelligence and machine learning using the Amazon Web Services (AWS) cloud service from the GSA Federal Acquisition Service (FAS) to classify pdf documents emailed into G-REX. The pilot built the pipeline to train individual document types that are typically uploaded in PDF format. The pilot was successful at illustrating how valuable time could be saved by completing this project in production, improving accuracy of document type assignment. The pilot utilized AWS FAS service, which would require additional funding. Appian announced they will provide the functionality as an out-of-the-box feature in late 2024, early 2025, to be able to scan pdf document types if G-REX can move to the cloud. The cloud assessment is ongoing as of submission of the need. This was an approved FY23 need.","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",N/A,N/A,Yes,"Yes – agency has access to source code, but it is not public.",N/A,Yes,I-SecTools,Less than 6 months,Yes,N/A,No,N/A,Yes,N/A,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,No,"Benchmark evaluation: Testing of the AI model has involved the use of benchmarks (either publicly available or internally created) to estimate performance in real-world settings, but has not been tested in an operational environment. ",There are no key risks currently as this system is designed to run beside a traditional human-operated SOC. All outputs are consumed by SOC operators and verified. No automated activity is taken by the AI beyond generating the initial alerts.,Planned or in-progress,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,N/A,In-person,N/A,Yes,No demographic data is collected by the SIEM tool. No demographic data fed to AI system for training or inference.,None of the above,N/A,"No – Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.","The AI system only generates alerts. All alerts are researched and validated by a human SOC analyst. Any action taken is at the discretion of the human, not the AI system.","No – This AI use case is not subject to the opt-out requirement because the AI functionality is solely used for the prevention, detection, and investigation of fraud or cybersecurity incidents, or the conduct of a criminal investigation.",Human's review alerts generated and only human's make actionable decisions Elastic Machine Learning Threat Detection - Phase 2,General Services Administration,GSA,GSA,Other,,None of the above.,"Leverage an LLM that has been given access to our security logs, to provide a more efficient search and discovery tool for SOC analysts. The benefit would be helping the SOC analyst find records of interest and/or associated logs quicker. It will also analyze logs automatically and suggest records of interest.",Text displayed on the user's screen,Initiated,Neither,Q4FY25,N/A,N/A,N/A,Developed with contracting resources.,N/A,No,Public Experience Portfolio – GSA ,Enable increased self-service through an expansion of digital costumer service tools,No,N/A,No,No,Yes,N/A,System and application logs collected into our SIEM tool,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",N/A,N/A,No,"Yes – agency has access to source code, but it is not public.",N/A,Yes,I-SecTools,Less than 6 months,Yes,N/A,No,N/A,Yes,N/A,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,No,No testing: No testing of the model to simulate performance in an operational environment has been conducted thus far.,No key risks identified. No PII or other demographic data fed into the AI system.,Planned or in-progress,No monitoring protocols have been established: Necessary infrastructure needed to perform monitoring of AI systems in production is not available and/or a plan to monitor models has not yet been established.,No - Some individual decisions or actions require direct human oversight.,N/A,In-person,N/A,Yes,No demographic data is collected by the SIEM tool. No demographic data fed to AI system for training or inference.,None of the above,N/A,"No – Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.",The AI system is designed to provide data to a human SOC analyst when queried. The AI initiates no actions on its own nor is its output used directly.,"No – This AI use case is not subject to the opt-out requirement because the AI functionality is solely used for the prevention, detection, and investigation of fraud or cybersecurity incidents, or the conduct of a criminal investigation.",Human's review alerts generated and only human's make actionable decisions Chatbot for Federal Acquisition Community,General Services Administration,GSA,FAS / GSA IT (IC),Other,,None of the above.,"The introduction of a chatbot will enable the GSA FAS NCSC to streamline the customer experience process, and automate providing answers to documented commonly asked questions through public facing knowledge articles. The end goal is this will reduce staffing requirements for NCSC’s live chat programs and allow the NCSC resources to be dedicated to other proactive customer services initiatives. Customers will still have the option to connect to a live agent if they choose by requesting an agent.",Text displayed on the user's screen,Initiated,Neither,Q4FY25,N/A,N/A,N/A,Developed with both contracting and in-house resources.,N/A,No,Public Experience Portfolio – GSA ,Enable increased self-service through an expansion of digital costumer service tools,No,N/A,No,No,Yes,N/A,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",N/A,N/A,No,No – agency does not have access to source code.,N/A,No,N/A,Less than 6 months,Yes,N/A,Yes,N/A,Yes,N/A,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,No,No testing: No testing of the model to simulate performance in an operational environment has been conducted thus far.,No key risks identified. No PII or other demographic data fed into the AI system.,Planned or in-progress,No monitoring protocols have been established: Necessary infrastructure needed to perform monitoring of AI systems in production is not available and/or a plan to monitor models has not yet been established.,No - Some individual decisions or actions require direct human oversight.,N/A,N/A - individuals are not interacting with the AI for this use case,N/A,No,N/A,None of the above,N/A,"No – Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.","Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.","No – This AI use case is not subject to the opt-out requirement because the AI functionality is solely used for the prevention, detection, and investigation of fraud or cybersecurity incidents, or the conduct of a criminal investigation.",N/A Document Workflow / Intelligent Data Capture and Extraction,General Services Administration,GSA,GSA IT (IC),Other,N/A,None of the above.,"GSA is driving towards a more accurate and scalable document workflow platform. GSA seeks to intelligently capture, classify, and transfer critical data from unstructured and structured documents, namely PDF files, to the right process, workflow, or decision engine.",N/A,Retired,Neither,N/A,N/A,N/A,11/1/2024,Developed with contracting resources.,N/A,No,Public Experience Portfolio – GSA ,Enable increased self-service through an expansion of digital costumer service tools,No,N/A,No,No,No,N/A,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",N/A,N/A,No,No – agency does not have access to source code.,N/A,No,N/A,Less than 6 months,Yes,N/A,No,N/A,Yes,N/A,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,No,No testing: No testing of the model to simulate performance in an operational environment has been conducted thus far.,N/A,Planned or in-progress,No monitoring protocols have been established: Necessary infrastructure needed to perform monitoring of AI systems in production is not available and/or a plan to monitor models has not yet been established.,No - Some individual decisions or actions require direct human oversight.,N/A,N/A - individuals are not interacting with the AI for this use case,N/A,No,N/A,None of the above,N/A,"No – Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.","Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.","No – This AI use case is not subject to the opt-out requirement because the AI functionality is solely used for the prevention, detection, and investigation of fraud or cybersecurity incidents, or the conduct of a criminal investigation.",N/A OAS Kudos Chatbot,General Services Administration,GSA,OAS / OCFO,Other,N/A,None of the above.,"The Office of Administrative Services would like to utilize a ChatBot to easily capture employee peer to peer recognitions. The OAS Kudos Chatbot leverages Natural Language Processing, inherent to Google Dialogflow, to facilitate a conversation between the AI software and the end user. This allows the chatbot to interpret and apply branching logic to what the user enters into the chat window.",N/A,Retired,Neither,N/A,N/A,N/A,11/1/2024,Developed with contracting resources.,N/A,No,Public Experience Portfolio – GSA ,Enable increased self-service through an expansion of digital costumer service tools,No,N/A,No,No,No,N/A,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",N/A,N/A,No,"Yes – agency has access to source code, but it is not public.",N/A,No,N/A,Less than 6 months,Yes,N/A,No,N/A,Yes,N/A,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,No,No testing: No testing of the model to simulate performance in an operational environment has been conducted thus far.,N/A,Planned or in-progress,No monitoring protocols have been established: Necessary infrastructure needed to perform monitoring of AI systems in production is not available and/or a plan to monitor models has not yet been established.,No - Some individual decisions or actions require direct human oversight.,N/A,N/A - individuals are not interacting with the AI for this use case,N/A,No,N/A,None of the above,N/A,"No – Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.","Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.","No – This AI use case is not subject to the opt-out requirement because the AI functionality is solely used for the prevention, detection, and investigation of fraud or cybersecurity incidents, or the conduct of a criminal investigation.",N/A NCMMS AI Chatbot,General Services Administration,GSA,"PBS, GSA IT (IC)",Other,Tech support for NCMMS users,None of the above.,"Provide inline tech support for NCMMS / IBM Maximo via a chatbot. This is an off-the-shelf product that provides natural language driven, contextual help using Maximo.","Chatbot responses to users questions about how to use NCMMS / Maximo, based on NCMMS training and support documents, videos, etc.",Acquisition and/or Development,Neither,N/A,7/1/2024,TODO,N/A,Developed with contracting resources.,47QFWA24F0012,No,Public Experience Portfolio – GSA ,Enable increased self-service through an expansion of digital costumer service tools,No,N/A,No,No,Other,We will train the system using our NCMMS program's training and support documents,We will train the system using our NCMMS program's training and support documents,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",N/A,N/A,No,No – agency does not have access to source code.,N/A,No,"OMNI Maximo Assistant. NCMMS is GSA's program name for IBM Maximo. OMNI is TRM's product name for their AI-enhanced inline help tool for Maximo",Less than 6 months,Yes,N/A,Yes,N/A,Yes,N/A,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,No,"Benchmark evaluation: Testing of the AI model has involved the use of benchmarks (either publicly available or internally created) to estimate performance in real-world settings, but has not been tested in an operational environment. ",N/A,Planned or in-progress,No monitoring protocols have been established: Necessary infrastructure needed to perform monitoring of AI systems in production is not available and/or a plan to monitor models has not yet been established.,No - Some individual decisions or actions require direct human oversight.,N/A,Online - in the terms or instructions for the service.,N/A,No,N/A. It's an inline tech support tool for NCMMS / IBM Maximo,Post-transaction customer feedback collections,N/A,Yes,N/A,Yes,N/A Login.gov,General Services Administration,GSA,FAS (TTS),Other,Facial matching for identity verification,None of the above.,"The purpose of Login.gov's identity verification service is to establish a link between the claimed identity and the subject creating a user account. The facial matching AI component of this service provides enhanced controls to protect users from identity fraud and the government from fraudulent access. The benefit to the user is to provide a fast, convenient way to prove their identity remotely and protect the public from identity theft. The benefit to the government is to that government partners may use this tool to verify the identities of people accessing government websites in a remote, unattended session and protect Government systems from fraudulent access.",The system delivers a match/no-match output.,Implementation and Assessment,"Rights-Impacting ",10/1/2023,4/12/2024,5/1/2024,N/A,Developed with contracting resources.,47QPCA24F0002,Yes,Public Experience Portfolio – GSA ,Enable increased self-service through an expansion of digital costumer service tools,No,N/A,No,Yes,Yes,N/A,"The model is not trained or tuned using agency data. GSA evaluated several facial-matching models in its Identity Proofing Performance Study using GSA data in a testing environment. More information about that study and the data it collected is available on this website and in GSA's Privacy Impact Assessment for the Study, which is located here under ""GSA Identity Proofing Equity Study"". In addition to in-depth studies like above, GSA upholds best product monitoring practices, such as evaluating success rates at each identity verification step, including facial matching. ","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",N/A,N/A,No,No – agency does not have access to source code.,N/A,Yes,"Login serves HISPs at VA, USDA, and SSA",Less than 6 months,Yes,N/A,Other,"Since this AI is a COTS SaaS solution, GSA does not currently have a process in place to train and develop the model. GSA will monitor its performance and include performance measures in contracts as needed.",Yes,N/A,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",Yes – Agency requested an extension for this use case.,Yes,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.","AI outputs that are biased in certain ways, including by being discriminatory or have a discriminatory effect. This was identified by agency personnel through literature reviews and initial testing of facial matching algorithms.",Yes – by another appropriate agency office that was not directly involved in the system’s development,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,N/A,None of the above,N/A,No,"To assess the potential risk of disparate outcomes, GSA is conducting the GSA Identity Proofing Performance Study (the “Study”) to understand how various vendors’ facial matching services perform across various demographics in uncontrolled environments similar to how this AI use case would operate. - The preliminary results of the Study have already been helpful in acquisition and design decisions to mitigate the risk of biased outcomes. For example, Login.gov requested that BPA Call Order vendors provide both internal and independent research on their products, including requesting the NIST FRTE 1:1 Demographic Differentials Summary that examines the false negative rate of 1:1 facial matching algorithms for different demographics, which includes testing the algorithms with photographs of people who are African or African American. - Over FY25, the Identity Proofing Performance Study team and academic partners will continue to evaluate the data and provide Login.gov with additional recommendations, which Login.gov will consider when maintaining and developing its service. For example, data from Login.gov’s operational identity-proofing process will be compared to similar datasets from the Study to develop and validate continuous performance indicators. ",Post-transaction customer feedback collections,N/A,Yes,N/A,Yes,N/A Gemini for Workspace,General Services Administration,GSA,GSA IT (IDE),Government Services (includes Benefits and Service Delivery),N/A,None of the above.,"The purpose of the Gemini for Workspace pilot study is to evaluate how advanced AI capabilities can enhance productivity, collaboration, and efficiency within the General Services Administration (GSA). Over a 90-day period, 200 users will interact with Gemini to assess its functionality and impact on daily operations. The system offers intelligent assistance for tasks like drafting documents and composing emails, enhances communication through smart scheduling and automated meeting summaries, provides data analysis and insights from large datasets, and automates routine administrative tasks to reduce manual workloads. All features available with Gemini Enterprise feature set will be made available within the test profile instance for pilot users. These features aim to help users accomplish productivity tasks more quickly and accurately, and purport to increase productivity across the agency. Enhanced communication tools are designed to facilitate better teamwork and coordination among staff, while access to advanced analytics could support data-driven decision-making. By automating routine tasks, employees can focus on higher-value activities, contributing further to operational efficiency and innovation. Access to the latest Gemini models can also support code development and debugging practices previously identified in earlier studies. The pilot will help GSA understand user engagement with AI tools and assess the practical benefits of integrating Gemini into the Google workspace. Users will only be permitted to use the features for non-sensitive use cases that involve public information and datasets. ","Feedback from this study will inform future AI investments and strategies, supporting GSA's objective to modernize its technology infrastructure and enhance the agency's effectiveness in fulfilling its mission.",Acquisition and/or Development,Neither,11/1/2024,11/1/2024,11/1/2024,N/A,Developed with contracting resources.,47HAA022F0129,No,Public Experience Portfolio – GSA ,Enable increased self-service through an expansion of digital costumer service tools,No,N/A,No,Yes,Yes,N/A,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",N/A,N/A,No,No – agency does not have access to source code.,N/A,No,N/A,Less than 6 months,Yes,N/A,No,N/A,Yes,N/A,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,No,No testing: No testing of the model to simulate performance in an operational environment has been conducted thus far.,N/A,Planned or in-progress,No monitoring protocols have been established: Necessary infrastructure needed to perform monitoring of AI systems in production is not available and/or a plan to monitor models has not yet been established.,No - Some individual decisions or actions require direct human oversight.,N/A,N/A - individuals are not interacting with the AI for this use case,N/A,No,N/A,None of the above,N/A,"No – Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.","Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.","No – This AI use case is not subject to the opt-out requirement because the AI functionality is solely used for the prevention, detection, and investigation of fraud or cybersecurity incidents, or the conduct of a criminal investigation.",N/A Test Fit Layouts,General Services Administration,GSA,GSA,Government Services (includes Benefits and Service Delivery),N/A,None of the above.,The purpose of the QBIQ pilot is to test the capabilities to rapidly test-fit spaces in consideration to evaluate whether AI can sufficiently be used to speed up vetting of space to fit requirements as well as utilize visualizations as a communication bridging tool with tenant agencies,Floor plan and 3D conceptual layouts,Acquisition and/or Development,Neither,4/1/2024,N/A,N/A,N/A,Developed with contracting resources.,One-time micro-purchase for 3 test fits,No,Public Experience Portfolio – GSA ,Enable increased self-service through an expansion of digital costumer service tools,No,N/A,No,Yes,No,N/A,Floor plan and 3D conceptual layouts,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",N/A,N/A,No,No – agency does not have access to source code.,N/A,No,qbiq.ai,Less than 6 months,Yes,N/A,No,N/A,Yes,N/A,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,No,"Benchmark evaluation: Testing of the AI model has involved the use of benchmarks (either publicly available or internally created) to estimate performance in real-world settings, but has not been tested in an operational environment. ",Exposure of floorplans,Planned or in-progress,No monitoring protocols have been established: Necessary infrastructure needed to perform monitoring of AI systems in production is not available and/or a plan to monitor models has not yet been established.,No - Some individual decisions or actions require direct human oversight.,N/A,N/A - individuals are not interacting with the AI for this use case,N/A,No,N/A,None of the above,N/A,"No – Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.","Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.",Yes,N/A Design Your Facility,Department of Health and Human Services,HHS,ACF,Mission-Enabling (internal agency support),,None of the above.,"The Office of Refugee Resettlement (ORR) manages many facilities for unaccompanied children and must regularly determine bed availability, configuration, and assignments. The current process for gathering this information takes lengthy training and assistance from system administrators. The Design Your Facility interface leverages LLMs to help users more quickly configure the correct building, room, and bed layout needed to create their facility’s “digital twin” in the UC Bed Network Facility Mapper without lengthy training or deskside assistance from system administrators.","The Design Your Facility tool uses large language models (LLMs) to transform building layout information shared by users in a conversational manner into data about buildings, rooms, and bed layers within the Unaccompanied Children (UC) Bed Network tooling. The AI model suggests object configurations that should be applied to correctly model a facility’s “digital twin” of the building, room, and bed layout. The workflow relies solely on user descriptions and user confirmations.",Acquisition and/or Development,Neither,2/1/2024,4/1/2024,,,Developed with contracting resources.,75D30122F15723,,,,,,,,No,,No training or fine-tuning; we are using secure commercially available LLMs.,Documentation has been partially completed,,,,,,Yes,Horizon (ACF instance of Palantir Foundry),Less than 6 months,No,,Yes,,No,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Discover Financial Business Intelligence System (FBIS) Report Analysis (Sub-CAN Line Items),Department of Health and Human Services,HHS,ACF,Mission-Enabling (internal agency support),,None of the above.,"This project provides a resource for ACF Discover users in the budget community to identify related line items across disparate reports from FBIS. The process searches the data and aggregates information (e.g. supplier name, document number, total obligations) to create an up-to-date, real-time spend plan that pulls from the open-closed report, requisition purchase order report, and integrates with funds distribution report. AI returns line items from the open-closed report and requisition purchase order report related to the sub-CAN line item description such as supplier name, document number, document type, and total obligations for that sub-CAN line item. The Discover FBIS Report Analysis (Sub-CAN Line Items) LLM helps find related obligations and line items from various reports and returns information crucial for a budget officer when they are creating a spend plan for their office. The LLM aids ACF Discover track various line items and pulls the information into one place: the Spend Plan module where ACF can track yearly budgets and monitor budget health.","A large-language model helps compile an aggregated report of costs by intaking a specific Congressional Appropriation Number (CAN), category, sub-CAN line item descriptions (which are not standardized), and a projection cost from the user (either by a CSV or manual entry).",Acquisition and/or Development,Neither,8/1/2024,9/1/2024,,,Developed with contracting resources.,75P00122A00010/75P00122F37003/P00004,,,,,,,,No,,No training or fine-tuning. HHS-FBIS reports are used as a data source for retrieval-augmented generation.,Documentation is complete,,,,,,Yes,ACF Discover (HHS instance of Palantir Foundry),Less than 6 months,No,,Yes,,Yes,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Grant Spend Health Analysis,Department of Health and Human Services,HHS,ACF,Mission-Enabling (internal agency support),,None of the above.,"The mission of the Office of Grants Management (OGM)is to provide high-quality fiscal stewardship and management for ACF-funded recipients across the country and in U.S. territories. This includes responsive oversight and technical assistance for discretionary awards, non-discretionary awards, and cooperative agreements. OGM seeks ways to prioritize proactive technical assistance to grant recipients based on likelihood that awarded funds may not be fully utilized. The goal is to identify still-active grants that are potentially on the path towards deobligating funding. This information would be visible to grant specialists within ACF. OGM anticipates developing customized training and technical assistance materials that can assist recipients if we predict that their grant is trending towards leaving funds unutilized.","As part of this project, we are conducting exploratory analyses including supervised and unsupervised AI techniques to determine whether the data we have on grant spend over time can power a predictive model that reasonably identifies grants with a higher likelihood of under-utilizing funds. ",Acquisition and/or Development,Neither,3/1/2024,3/1/2024,,,Developed with contracting resources.,75ACF123F80045,,,,,,,,Yes,,Grant financial data from usaspending.gov and the Payment Management System. Recipient data from SAM.gov.,Documentation is widely available,,,,,,No,,Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, Structuring and Validating Completeness of Case Data Information,Department of Health and Human Services,HHS,ACF,Mission-Enabling (internal agency support),,None of the above.,"In accordance with the Ms L vs ICE settlement, the U.S. Customs and Border Protection (CBP) must provide specific pieces of information about family separations. Beginning November 2024, CPB will release this information. ACF's Office of Refugee Resettlement (ORR) plans to add family separation information to existing child profile information for reference. The AI output structures free form data into a more easily searchable format. These data will be used to assist in the placement process.","Creates a structured data asset of the critical data points needed about a separation case. AI will be used to conduct initial parsing of the data provided by CBP and highlight whether or not the required fields from the Ms L vs. ICE settlement are included and can therefore be updated into the child's profile in ORR's data system. ORR's Intakes Team does final review and in cases where data appears to be missing, the ORR Intakes Team reaches back out to CBP for that information.",Acquisition and/or Development,Safety-Impacting,8/1/2024,11/1/2024,,,Developed with contracting resources.,75D30122F15723,,,,,,,,No,,No training or fine tuning; we are prompting secure commercially-available LLMs.,Documentation has been partially completed,,,,,,Yes,Horizon (ACF instance of Palantir Foundry),Less than 6 months,No,,Yes,,No,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, Triaging Notice of Concern Submissions,Department of Health and Human Services,HHS,ACF,Government Services (includes Benefits and Service Delivery),,None of the above.,"The Office of Refugee Resettlement (ORR) has amassed a large backlog of Notice of Concern (NOC) PDF forms that contain critical information regarding safety of children who have left ORR's care. The AI-powered triage tool will support Office of Refugee Resettlement staff more effectively and efficiently review NOCs by helping structure and prioritize received NOCs. Through more effective triage, ORR will begin to cut down on the large backlog that has accumulated.","In the NOC triage system, AI will be used for two purposes: 1. We will use AI to parse the NOCs to generate structured fields from the text and validate categorizations. 2. AI will provide prioritization recommendations -- but not make decisions -- on the most critical NOCs for the ORR team to review first. ",Acquisition and/or Development,Safety-Impacting,6/1/2024,6/1/2024,,,Developed with contracting resources.,75D30122F15723,,,,,,,,No,,No training or fine tuning. Notional NOCs were used to prompt engineer secure commercially-available LLMs and evaluate performance. End-user testing was used to evaluate on real NOCs.,Documentation has been partially completed,,,,,,Yes,Horizon (ACF instance of Palantir Foundry),Less than 6 months,No,,Yes,,No,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Unaccompanied Children Program Policy & Procedure Research Tool,Department of Health and Human Services,HHS,ACF,Mission-Enabling (internal agency support),,None of the above.,"The Office of Refugee Resettlement (ORR) conducts monitoring visits at least monthly to ensure that care providers meet minimum standards for the care and timely release of unaccompanied children, and that they abide by all Federal and State laws and regulations, licensing and accreditation standards, ORR policies and procedures, and child welfare standards. If ORR monitoring finds a care provider to be out of compliance with requirements, ORR issues corrective action findings and requires the care provider to resolve the issue within a specified time frame. Compliance determination involves research into the various laws, standards, policies, and procedures. The UC Program Policy & Research Tool assists the UC monitoring team with research that in many cases take a long time. The goal of the UC Program Policy & Research Tool is to speed up this research as children's health and well-being may be impacted before a corrective action finding is issued and the issue is resolved.","The UC Program Policy & Procedure Research Tool speeds up research of relevant laws, standards, policies, and procedures content curated and approved by ORR's policy team. This research is one part of the process that informs ORR's monitoring team's decisions on whether corrective actions are needed and if so, what corrective actions. AI is not used to suggest corrective actions but rather support determination of whether care providers are in compliance.",Acquisition and/or Development,Neither,10/1/2024,11/1/2024,,,Developed with contracting resources.,75D30122F15723,,,,,,,,Yes,,No training or fine-tuning. ORR-approved policy content are used as a data source for retrieval-augmented generation.,Documentation is complete,,,,,,Yes,Horizon (ACF instance of Palantir Foundry),More than 12 months,No,,Yes,,Yes,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Unaccompanied Children Process Model Digital Twins,Department of Health and Human Services,HHS,ACF,Mission-Enabling (internal agency support),,None of the above.,"The Office of Refugee Resettlement (ORR) Unaccompanied Children Bureau provides a safe and appropriate environment to children who enter the United States without immigration status and without a parent or legal guardian who is able to provide for their physical and mental well-being (referred to as “unaccompanied children”). ORR funds residential care provider facilities that provide temporary housing and other services to unaccompanied children in ORR custody. ORR and its care providers work to ensure that children are released timely and safely from ORR custody to parents, other family members, or other adults (often referred to as “sponsors”) who can care for the child’s physical and mental well-being. Unaccompanied children remain in ORR’s care and custody until they are released to a parent or other sponsor in the United States, are repatriated to their home country, obtain legal status, or turn 18 years old, at which time they are transferred to the custody of DHS. There are many processes involved throughout. The goal is for ORR to be more proactive in strategic planning and contingency planning.","ORR is exploring use of process digital twins to provide the ability to run ""what if"" scenarios",Initiated,Neither,10/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Analyzing Public Comments on Proposed Rule,Department of Health and Human Services,HHS,ACF,Mission-Enabling (internal agency support),,None of the above.,"In the federal rulemaking process, the public has an opportunity to provide comments on proposed new regulation. AI was used to support the public comment review process for a proposed rule to make quality improvements in the Head Start program. The public comment analysis pipeline provided an Excel spreadsheet of comment segments that were tagged and Word files of topic-based summary drafts. Subject matter experts from the Office of Head Start reviewed comments by topic and further analyzed and refined the topic-based summaries. The benefit of using AI for this purpose is improved baseline information following public comment to support more expeditious final rule development.","A large language model (LLM) was used to tag public comments by: topics selected primarily by policy experts; sentiment; and intent (e.g. question, suggestion) alongside first drafts of topic-based summaries. ",Retired,Neither,,,,2/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Outreach List Segmentation,Department of Health and Human Services,HHS,ACF,Mission-Enabling (internal agency support),,None of the above.,"The Office of External Affairs (OEA) is responsible for sharing information from ACF with external constituents. OEA had a long list of constituent email addresses that were without context. They wanted the ability to send targeted emails to different groups of people whose organizations work in particular policy spaces. The AI tool returned a spreadsheet of the provided domains with structured information on likely organization name, summary of their mission, policy domains (e.g. public health, education), and website. This spreadsheet allows OEA to segment their lists and target organizations more efficiently.","An AI-enabled search tool was used to research likely organization name, purpose, and policy space based on the domains (e.g. ""@organization.com"") from contact email addresses.",Retired,Neither,,,,8/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Ask HR Policy,Department of Health and Human Services,HHS,ACF,Mission-Enabling (internal agency support),,None of the above.,"This project provides a resource for ACF Executive Officers/Administrative Officers and managers to easily navigate Human Resources (HR) Policy documentation and quickly get answers to their questions that are based on the official HR Policy Library that is on the HHS.gov website (https://www.hhs.gov/about/agencies/asa/ohr/hr-library/index.html) Ask HR Policy provides a secure interface permissioned only to select ACF Executive Officers and Administrative Officers. Users type in questions about managing employees covered by the HR Policy Library, and Ask HR Policy provides a narrative of its “thought process” then suggests an answer based in the documentation, alongside links that take the user to the relevant section of the official document.",We are using LLMs to analyze the questions inputted by users. The AI model synthesizes the provided documentation sources and then summarizes and answers the users questions with cited sources.,Operation and Maintenance,Neither,5/1/2024,5/1/2024,5/1/2024,,Developed with contracting resources.,75P00122A00010/75P00122F37003/P00004,,,,No,,No,No,No,,No training or fine-tuning. HHS HR Policies were used as a data source for retrieval-augmented generation.,Documentation is complete,,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,ACF Discover (HHS instance of Palantir Foundry),Less than 6 months,No,,Yes,,Yes,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Child Welfare Information Gateway OneReach Application,Department of Health and Human Services,HHS,ACF,Government Services (includes Benefits and Service Delivery),,None of the above.,"The Childrens Bureau runs the Children Welfare Information Gateway, a connection to trusted resources on the child welfare continuum. The Information Gateway has a hotline for answering questions or requesting information: https://www.childwelfare.gov/stay-connected/contact/ The Information Gateway hotline automates some of the more routine questions and requests, allowing staff to focus on more complex, nuanced situations.","The Information Gateway hotline connects to a phone interactive voice response (IVR) managed by OneReach AI. OneReach maintains a database of state hotlines for reporting child abuse and neglect that it can connect a caller to based on their inbound phone area code. Additionally, OneReach offers a limited FAQ texting service that utilizes natural language processing to answer user queries. User queries are used for reinforcement training by a human AI trainer and to develop additional FAQs.",Operation and Maintenance,Neither,3/1/2020,4/1/2020,6/1/2020,,Developed with contracting resources.,20EMPO0106,,,,Yes,This use case is operated in compliance with HHS's agencywide Information Quality Act policy and procedures.,Yes,No,No,,N/A,Documentation is widely available,,,No,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,No,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, Collective Bargaining Compass,Department of Health and Human Services,HHS,ACF,Mission-Enabling (internal agency support),,None of the above.,"This project provides a resource for ACF managers to easily navigate Collective Bargaining Agreement documentation and quickly get answers to their questions that are based on the official 400-page rulebook. The Collective Bargaining Compass provides a secure Virtual Assistant interface permissioned only to select ACF managers. Users type in questions about managing employees covered by the Collective Bargaining Agreement, and the Virtual Assistant provides a narrative of its “thought process” then suggests an answer based in the documentation, alongside links that take the user to the relevant section of the official document.","We are using LLMs to analyze the questions inputted by users, which then the AI model synthesizes the provided documentation sources and then summarizes and answers the users questions with cited sources.",Operation and Maintenance,Neither,2/1/2024,2/1/2024,3/1/2024,,Developed with contracting resources.,75P00122A00010/75P00122F37003/P00004,,,,No,,No,No,No,,No training or fine-tuning. HHS-The NTEU 2023 Collective Bargaining Agreement was used as a data source for retrieval-augmented generation.,Documentation is complete,,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,ACF Discover (HHS instance of Palantir Foundry),Less than 6 months,No,,Yes,,Yes,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Discover User Assistant,Department of Health and Human Services,HHS,ACF,Mission-Enabling (internal agency support),,None of the above.,"This project provides a resource for ACF Discover Staff Management Module users (i.e. ACF Executive Officers and other ACF administrative and management staff) to easily navigate ACF Discover Staff Management platform and instructional documentation and quickly get answers to their questions about the module’s features. The User Documentation Assistant provides a secure virtual assistant interface that is only available to ACF Discover Users. Users are able to ask the assistant specific questions about the capabilities of ACF Discover along with how to leverage tools and applications. The Virtual Assistant is able to provide answers by referencing the User Reference guide.","We are using LLMs to analyze the questions inputted by users, which then the AI model synthesizes the provided documentation sources and then summarizes and answers the users' questions with cited sources.",Operation and Maintenance,Neither,1/1/2024,1/1/2024,1/1/2024,,Developed with contracting resources.,75P00122A00010/75P00122F37003/P00004,,,,No,,No,No,No,,No training or fine-tuning. HHS-The Discover Staff Management Module User Reference Guide was used as a data source for retrieval-augmented generation.,Documentation is complete,,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,ACF Discover (HHS instance of Palantir Foundry),Less than 6 months,No,,Yes,,Yes,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Policy Knowledge Base Data Migration,Department of Health and Human Services,HHS,ACF,Mission-Enabling (internal agency support),,None of the above.,"The Office of Family Assistance (OFA) provides technical assistance to grantees, including answering questions about policy regarding the Temporary Assistance for Needy Families (TANF) program. Historically, OFA has tracked its accumulated TANF knowledge across disparate locations and file types. Given the quantity of knowledge base articles involved in this project (>1,100), the AI draft article topics and categories helped speed up the creation of the knowledge base minimum viable product.","As part of a broader effort to establish a TANF knowledge base on a unified platform, large language models were used to: 1) generate draft knowledge base article topics based on historical tracker information; and 2) categorize articles to help organize the knowledge base.",Operation and Maintenance,Neither,6/1/2024,6/1/2024,9/1/2024,,Developed with both contracting and in-house resources.,75ACF123F80045,,,,No,,No,No,No,,Historical trackers of questions and answers to TANF policy questions,Documentation has been partially completed,,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, Qualitative analysis,Department of Health and Human Services,HHS,ACF,Mission-Enabling (internal agency support),,None of the above.,"ACF staff often conduct surveys and interviews, which generate qualitative data that needs to be analyzed for themes and trends. The AI functionality assists users in identifying relevant passages in their narrative data related to specific topics.",We use commercial off-the-shelf software that have built-in AI functionality to suggest topic tagging of qualitative data.,Operation and Maintenance,Neither,3/1/2023,3/1/2023,9/1/2023,,Developed in-house.,,,,,No,,Yes,No,No,,N/A,Documentation is complete,,,No,No – agency does not have access to source code.,,No,,6-12 months,No,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, AHRQ Search,Department of Health and Human Services,HHS,AHRQ,Mission-Enabling (internal agency support),,None of the above.,"Organization wide search that includes Relevancy Tailoring, Auto-generation Synonyms, Automated Suggestions, Suggested Related Content, Auto Tagging, and ""Did you mean?"" to allow visitors to find specific content. This AI use case enhances our agency's efficiency and user experience by optimizing search results, auto-completing queries, suggesting relevant searches and content tags, as well as proposing spelling corrections. ","This AI use case aims to optimize search results by adjusting their ranking, ensuring that the most pertinent information is displayed at the top. It also enhances search effectiveness by adding synonyms to queries behind the scenes. It further improves user experience by auto-completing queries as they are being typed, and showing related searches that might offer additional valuable insights. The system also proposes content tags automatically, leveraging machine learning to assess existing content tagging patterns. Additionally, it suggests spelling corrections and reformats search queries based on data from Google Analytics.",Operation and Maintenance,Neither,9/1/2019,9/1/2019,10/1/2020,,Developed with contracting resources.,75Q80121A00001,,,,No,,No,No,No,,N/A,Documentation is missing or not available,,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,AHRQ AWS Enclave,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Chatbot,Department of Health and Human Services,HHS,AHRQ,Mission-Enabling (internal agency support),,None of the above.,"This use case introduces a conversational interface that enables users to ask questions about AHRQ content using natural language. This will serve as a modern, efficient replacement for the traditional public inquiry telephone line. Improved user experience and reduced resources responding to public inquiries",An interactive interface that can respond to plain language queries in real time using natural language processing,Operation and Maintenance,Neither,10/1/2020,10/1/2020,2/1/2022,,Developed with contracting resources.,75Q80121A00001,,,,No,,No,No,No,,N/A,Documentation is missing or not available,,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,AHRQ AWS Enclave,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Chatbot,Department of Health and Human Services,HHS,AHRQ,Mission-Enabling (internal agency support),,None of the above.,"For the Patient Safety Organization Privacy Protection Center website, this use case provides an interface that allows users to conversationally ask questions about content in natural language. Improved user experience and reduced resources responding to public inquiries",An interactive interface that can respond to plain language queries in real time using natural language processing,Initiated,Neither,8/1/2021,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Enhancing Diversity in Peer Review - Pilot,Department of Health and Human Services,HHS,AHRQ,Mission-Enabling (internal agency support),,None of the above.,"The Division of Scientific Review (DSR) is aiming to enhance diversity in AHRQ peer review groups by revising recruitment strategies, aiming for 20% representation from underrepresented minorities in each study section. This process, involving networking, outreach, and various resources, typically takes at least three months. By utilizing AI to swiftly and accurately identify diverse and qualified grant reviewers, our agency will foster a more efficient process for identifying scientific reviewers and a more equitable and inclusive scientific review process. This will help us achieve our goal of providing safer, high-quality, accessible, and affordable healthcare to individuals and communities throughout the U.S., thereby improving the overall effectiveness of our agency.","This AI use case will involve training an artificial intelligence system to examine the background of peer reviewers - taking into account factors such as expertise, race/ethnicity, gender, and geographic area - and match these inputs to the relevant study section, subsequently referring the reviewer to the Scientific Review Officer (SRO). This flexible system will empower SROs to make the final recruitment decision or to refer the expert to another study section or Special Emphasis Panel (SEP).",Initiated,Neither,6/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, NLQuery- As Data or Pulse,Department of Health and Human Services,HHS,AHRQ,Mission-Enabling (internal agency support),,None of the above.,"As a feature to clinical dashboards available on a secured page on the Patient Safety Organization Privacy Protection Center website for PSOs who submit data, provides capability to allow user to type a question in common language and instantly get a response. Answers come in the form of automatic data visualizations, with no need to manually drag-and-drop fields or understand the nuances of user's data’s structure. Improved functionality of the clinical dashboards. Potential to encourage additional reporting of data in light of benefit to user.","Provide capability to allow user to type a question in common language and instantly get a response . Answers come in the form of automatic data visualizations, with no need to manually drag-and-drop fields or understand the nuances of user's data’s structure.",Initiated,Neither,8/1/2021,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Quality and Safety Review System AI-enabled automated abstraction,Department of Health and Human Services,HHS,AHRQ,Mission-Enabling (internal agency support),,None of the above.,"The aim of this AI use case is to enhance or eliminate the need for manual abstraction of patient records and manual data entry into the QSRS tool, thereby streamlining the data entry process. This AI use case has the potential to significantly help by increasing the volume of cases abstracted while maintaining or lowering government expenditure, thereby enabling more work to be done at the same or reduced cost. Additionally, it can maintain the number of cases abstracted with improved efficiency, thus also reducing government costs by achieving the same output at a lower cost. Furthermore, it will improve abstraction accuracy and expand AI/ML capabilities at both the national and local hospital levels while ensuring data quality improvements so there is trusted and reliable data for reporting purposes.","This AI use case is designed to automate the data abstraction process once the OCR system has transformed PDF images into text. It also aims to automate data entry into the Quality and Safety Review System (QSRS), thereby increasing efficiency and accuracy.",Acquisition and/or Development,Neither,6/1/2024,10/1/2024,,,Developed with contracting resources.,75Q80122F80006,,,,,,,,No,,N/A,Documentation is missing or not available,,,,,,Yes,AHRQ AWS Enclave,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Document Q&A Interface ,Department of Health and Human Services,HHS,ASPR,Law & Justice,,None of the above.,"The SCCT consolidates a number of documents for every response that it monitors, and often analysts have questions about response context stored in those documents Increase an analyst / user on Protect's knowledge of a specific response or MCM by allowing them to ""ask questions"" of associated / uploaded SCCT media",Large Language Models are used to query existing documents for answers based on a user's query,Acquisition and/or Development,Neither,7/1/2024,8/1/2024,,,Developed with contracting resources.,000HCVL1-2022-66353,,,,,,,,No,,SCCT Distributor Data,Documentation is complete,,,Yes,No – agency does not have access to source code.,,No,,Less than 6 months,No,,Yes,,No,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Emergency Inventory Redistribution Model,Department of Health and Human Services,HHS,ASPR,Emergency Management,,None of the above.,"Optimization models are needed to identify efficient preparedness and responses strategies for inventory management and distribution during an emergency. This can be used to identify possible interventions.",This linear programming (constrained optimization) model can identify the optimal strategy for meeting demand for medical supplies during an emergency event.,Implementation and Assessment,Neither,8/1/2023,8/1/2023,,,Developed with contracting resources.,75A50122C00052,,,,No,,No,No,Yes,,SCCT Distributor Data,Documentation has been partially completed,,,Yes,No – agency does not have access to source code.,,No,,Less than 6 months,No,,Yes,,No,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, emPOWER AI,Department of Health and Human Services,HHS,ASPR,Health & Medical,,None of the above.,"Public health authorities, first responders and others across the emergency management spectrum indicated that they had to be able to more rapidly access emPOWER publicly available data, particularly in unstable internet conditions, during disasters. emPOWER AI, an Amazon Alexa Skill, was created to allow anyone with a smartphone to be able to request the data and informational resources. First responders, public health authorities, emergency managers, health care providers, and other community partners can use emPOWER AI prior to, during, and after an incident, emergency, or disaster to anticipate, plan for, and respond to the access and functional needs of at-risk populations in their communities using publicly available emPOWER data, informational resources and training.","emPOWER AI gives the user publicly available data from the HHS emPOWER Map on the number of electricity-dependent Medicare beneficiaries at the national, state, territory, county, and ZIP Code levels. It also provides information on other HHS emPOWER Program tools and resources that collectively support emergency preparedness, response, recovery, mitigation, and resilience activities. ",Operation and Maintenance,Neither,1/1/2018,9/1/2017,1/1/2019,,Developed with contracting resources.,GSA Schedules,,,,Yes,,No,No,No,,None,Documentation is complete,,,No,"Yes – agency has access to source code, but it is not public.",,Yes,Not Reported,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, Entity Resolution Tool,Department of Health and Human Services,HHS,ASPR,Mission-Enabling (internal agency support),,None of the above.,"SCCT receives order information that references thousands of manufacturers and customers. These data need to be standardized. This allows analysts to aggregate data by manufacturer or customer to better characterize order and delivery patterns.","This tool uses embedding models, hierarchical clustering, and large language models to identify groups of duplicate entity names and assign a new standardized name.",Implementation and Assessment,Neither,6/1/2024,7/1/2024,,,Developed with contracting resources.,75A50122C00052,,,,No,,No,No,Yes,,SCCT Distributor Data,Documentation has been partially completed,,,Yes,No – agency does not have access to source code.,,No,,Less than 6 months,No,,Yes,,No,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Executive Report Generation AI Copilot ,Department of Health and Human Services,HHS,ASPR,Mission-Enabling (internal agency support),,None of the above.,"SCCT produces executive summaries and commercial product awareness reports (CPARs) on supply chain disruptions, weather events, shortages, etc. These utilize a large amount of SCCT data that needs to be summarized and explained in detail. Used to decrease the amount of time needed to produce an executive report, reducing the need for SCCT analysts to create them from scratch every time",Large language models are utilized to summarize existing documents and data in order to produce the base text for an executive report. ,Operation and Maintenance,Neither,2/1/2024,2/1/2024,3/1/2024,,Developed with contracting resources.,000HCVL1-2022-66353,,,,No,,No,No,Yes,,SCCT Distributor Data,Documentation has been partially completed,,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,No,,Yes,,No,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Flow Model Edge Prioritization Estimation Tool,Department of Health and Human Services,HHS,ASPR,Emergency Management,,None of the above.,"This model is needed to investigate potential impacts of supply disruptions. This is one component in a simulation tool used to quantify the impact of supply chain disruptions on customers.",This model uses linear regression to summarize patterns in preferential order fulfillment and predict future fill rates.,Operation and Maintenance,Neither,5/1/2023,9/1/2023,8/1/2024,,Developed with contracting resources.,75A50122C00052,,,,No,,No,No,Yes,,SCCT Distributor Data,Documentation has been partially completed,,,Yes,No – agency does not have access to source code.,,No,,Less than 6 months,No,,Yes,,No,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Generalized Demand Model Sensitivity Analyzer,Department of Health and Human Services,HHS,ASPR,Mission-Enabling (internal agency support),,None of the above.,"The generalized demand model is used by SCCT to create demand projections for critical medical products. This is a diagnostic tool used to help analysts understand and refine their models.",This constrained regression model is used to identify the primary drivers of uncertainty in demand models and quantify their impact.,Operation and Maintenance,Neither,9/1/2023,12/1/2023,2/1/2024,,Developed with contracting resources.,75A50122C00052,,,,No,,No,No,Yes,,None,Documentation has been partially completed,,,Yes,No – agency does not have access to source code.,,No,,Less than 6 months,No,,Yes,,No,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Google Search Model Alerts,Department of Health and Human Services,HHS,ASPR,Mission-Enabling (internal agency support),,None of the above.,"Publicly available information can enhance the ability to identify supply chain disruptions. This provides automated alerts that can help trigger review of the relevant data by analysts.","This model, based on exponentially weighted moving averages, identifies rising trends in timeseries related to google search volume for keywords of interest.",Operation and Maintenance,Neither,1/1/2023,6/1/2023,4/1/2024,,Developed with contracting resources.,75A50122C00052,,,,No,,No,No,Yes,,None,Documentation has been partially completed,,,Yes,No – agency does not have access to source code.,,No,,Less than 6 months,No,,Yes,,No,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, Google Search Model Rising Trend Entity Extraction,Department of Health and Human Services,HHS,ASPR,Mission-Enabling (internal agency support),,None of the above.,"Publicly available information can enhance the ability to identify supply chain disruptions. The list of products produced by this model helps analysts identify products that may be in shortage.",This tool uses language models to identify the object in shortage from a list of shortage related rising search terms identified by Google.,Operation and Maintenance,Neither,1/1/2023,6/1/2023,4/1/2024,,Developed with contracting resources.,75A50122C00052,,,,No,,No,No,Yes,,None,Documentation has been partially completed,,,Yes,No – agency does not have access to source code.,,No,,Less than 6 months,No,,Yes,,No,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, Manufacturer Name Standardization,Department of Health and Human Services,HHS,ASPR,Mission-Enabling (internal agency support),,None of the above.,"The SCCT receives many manufacturer name variants from its many distributors that need to be consolidated and standardized This allows analysts to have more standardized manufacturer names for reporting","An advanced grouping algorithm groups similar manufacturer names together, and a Large Language Model recommends a standardized name based on all variants",Operation and Maintenance,Neither,4/1/2024,4/1/2024,8/1/2024,,Developed with contracting resources.,75A50122C00052,,,,No,,No,No,Yes,,SCCT Distributor Data,Documentation has been partially completed,,,Yes,No – agency does not have access to source code.,,No,,Less than 6 months,No,,Yes,,No,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Model based open source reporting search and summarization,Department of Health and Human Services,HHS,ASPR,Law & Justice,,None of the above.,"SCCT collects open source information for monitoring supply chain events. Language models provide an initial summary that analysts can use to select topics of interest in order to focus on relevant events.",Large language models are used to search open source information and summarize topics related to supply chain events.,Initiated,Neither,8/1/2024,,,,Developed with contracting resources.,000HCVL1-2022-66353,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, PPE Product Classification Model,Department of Health and Human Services,HHS,ASPR,Mission-Enabling (internal agency support),,None of the above.,"SCCT receives information about thousands of products from distributors. This is used to predict if the product falls into one of the following 7 categories: 1) Gloves – Surgical & Exam, 2) Needles & Syringes, 3) Gowns – Isolation & Surgical, 4) Masks – N95, 5) Masks – Surgical/Procedural, 6) Eye / Face Shields, 7) Test Kits. These classifications are used to aggregate data in order to make it possible to analyze trends.",This model uses bayesian anomaly detection and random forests to assign products to a list of predefined product classifications.,Operation and Maintenance,Neither,1/1/2023,1/1/2023,1/1/2023,,Developed with contracting resources.,000HCVL1-2022-66353,,,,No,,No,No,Yes,,SCCT Distributor Data,Documentation has been partially completed,,,Yes,No – agency does not have access to source code.,,No,,6-12 months,No,,Yes,,No,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Adverse Childhood Experiences (ACEs) Literature Review Dashboard,Department of Health and Human Services,HHS,CDC,Mission-Enabling (internal agency support),,None of the above.,"This solution will impact the process of conducting literature reviews for scientific research related to ACEs. Identifying relevant articles in a literature review is a time intensive process. Utilizing large language models, this system enables quick identification of relevant articles to be included in a ACEs literature database. This database allows ACEs researchers within CDC's National Center for Injury Prevention and Control a quick reference to develop hypotheses and identify gaps in the ACEs literature",This solution automatically determines the relevancy of published journal articles. Relevant articles are included in a master dataset that is linked to a PowerBI,Acquisition and/or Development,Neither,2/1/2024,3/1/2024, , ,Developed with contracting resources.,N/A, , , , , , , ,No,,Data used are published journal articles which may or may not be owned by CDC but may be accessed either through freely accessible sources or specific agreements CDC has to research articles.,Documentation is missing or not available,,, , , ,Yes,Enterprise Data Analytics and Visualization (EDAV) Platform,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, Automated Analysis of Injury Control Research Center (ICRC) Annual Progress Reports (APRs) using Large Language Models,Department of Health and Human Services,HHS,CDC,Mission-Enabling (internal agency support),,None of the above.,"This AI affects the process of reviewing and analyzing the Annual Progress Reports (APRs) submitted by Injury Control Research Centers (ICRCs). Specifically, the AI will focus on identifying and analyzing text sections of the APRs that detail reported challenges, as well as other text-heavy sections in future stages of the project. The AI is designed to streamline the review process, improve efficiency, and support the evaluation of the performance and progress of ICRC-funded activities. The output of the AI will help to quickly and efficiently identify key challenges and insights from ICRC APRs, enabling more effective decision-making in the review process. By automating the extraction and analysis of critical information, the AI allows the ICRC team to focus on higher-level evaluation and strategic planning. The AI will help reduce the time and resources needed for manual review, improve the consistency and accuracy of assessments, and facilitate faster responses to ICRC needs. Ultimately, this will support ICRCs in overcoming challenges and achieving their research and injury control goals, benefiting the public health system as a whole.","The AI analyzes the textual content of APRs. We have started our analysis by focusing on the sections detailing the challenges faced by ICRCs during the 2019 funding cycle. The AI identifies key themes, trends, and critical information that may require further attention. The AI methodology extracts insights and patterns from the data, which can then be compared with manual qualitative analysis outcomes. In subsequent stages, the AI will be expanded to analyze other sections of the APRs, such as progress toward goals and program impact.",Acquisition and/or Development,Neither,8/1/2023,12/1/2023, , ,Developed in-house., , , , , , , , ,No,,Injury Control Research Center Annual Progress Reports,Documentation is widely available,,, , , ,Yes,Enterprise Data Analytics and Visualization (EDAV) Platform,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Detecting Stimulant and Opioid Misuse and Illicit Use,Department of Health and Human Services,HHS,CDC,Health & Medical,,None of the above.,"Federal Health Survey analysis. The machine learning models created in this project will have an impact on the statistical analysis of electronic health record (EHR) data gathered through the (NHCS) National Hospital Care Survey. Furthermore, when the model is made available to the public, it may also influence the analysis of other user datasets that contain EHR clinical notes. Certain non-therapeutic models do not have corresponding medical codes available in clinical notes and the information is only available in free-text fields. The machine learning models help to discover non-therapeutic drug use as described in EHR clinical notes which are novel insights previously impossible to gleam from EHRs without AI.","Two machine learning models will be created, one for internal use and another for public release. In both instances, these models will be utilized alongside rule-based text analysis to ascertain whether a patient in a hospital setting has used a drug therapeutically (in accordance with medical prescription) or non-therapeutically. This information is used to gather new insights from EHRs for health statistics published through the NHCS.",Acquisition and/or Development,Neither,3/1/2024,9/1/2024, , ,Developed in-house., , , , , , , , ,No,,National Hospital Care Survey 2020 clinical notes,Documentation has been partially completed,,, , , ,No, ,More than 12 months,Yes,,Yes,,No,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, DHP Virtual Assistant,Department of Health and Human Services,HHS,CDC,Mission-Enabling (internal agency support),,None of the above.,"Literature Review for HIV research. The AI virtual assistant will help HIV researchers with information retrieval on information related to HIV and HIV research. Our AI virtual assistant will help increase productivity among our HIV researchers by retrieving information related to HIV and HIV research.",This AI assistant will use retrieval augmented generation (RAG) to return information related to HIV based on a user's query and other documents.,Acquisition and/or Development,Neither,11/1/2024,11/1/2024, , ,Developed in-house., , , , , , , , ,Yes,,"Internal documentation containing mapped eHARS LOINC codes, IQVIA data dictionaries, APRs",Documentation is complete,,, , , ,Yes,Enterprise Data Analytics and Visualization (EDAV) Platform,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Fuzzy matching tool,Department of Health and Human Services,HHS,CDC,Mission-Enabling (internal agency support),,None of the above.,"Clearance and review management of CDC publication. CDC's Office of Science (OS) manages the eClearance information system, which is responsible for tracking the submission of CDC-authored journal manuscripts and Morbidity and Mortality Weekly Reports (MMWRs) for CDC's internal peer review process. This tool will help ensure CDC-authored publications have completed the National Institutes of Health Manuscript Submission (NIHMS) process and are compliant with CDC's public access to publications policy. It will also be used to help centers and divisions identify publications from their organization for science prioritization and impact analyses.","The fuzzy matching tool uses pre-trained, open source LLMs to identify similar records between eClearance submissions and CDC-authored publications. This tool is intended for internal use only.",Acquisition and/or Development,Neither,9/1/2022,3/1/2023, , ,Developed in-house., , , , , , , , ,No,,eClearance submissions data and Science Clips data,Documentation is complete,,, , , ,No, ,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Global (GIB) Initiative for Influenza Vaccine Equity (GIIVE): Ensuring Universal Access,Department of Health and Human Services,HHS,CDC,Health & Medical,,None of the above.,"Literature Review The output is a list of abstracts that may have related information to vaccine inequity. These identified abstracts will be reviewed ""by-hand"" and then those associated publications will be reviewed by epidemiologists further.","The Through APIs, large language models are used to review a large number of abstracts to help identify which kinds of groups may be least likely to obtain vaccines in the global space--marginalized groups within country, for example.",Acquisition and/or Development,Neither,10/1/2024,10/1/2024, , ,Developed in-house., , , , , , , , ,No,,Data used are published journal articles which may or may not be owned by CDC but may be accessed either through freely accessible sources or specific agreements CDC has to research articles.,Documentation has been partially completed,,, , , ,Yes,Enterprise Data Analytics and Visualization (EDAV) Platform,Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, HIV Data Virtual Assistant,Department of Health and Human Services,HHS,CDC,Mission-Enabling (internal agency support),,None of the above.,"The AI will impact the data retrieval and automation process for HIV researchers, specifically in providing information from datasets and potentially generating code for desired variable names and analysis or generating information about a particular dataset. This solution aims to alleviate the challenges faced by scientists searching for the appropriate HIV-related data to address their inquiries. This will improve productivity and research efforts.","This AI assistant will use retrieval augmented generation (RAG) to return information related to HIV based on a user's query. Our aim is to input all dataset documentation and metadata into an index with the goal of creating a virtual assistant. This assistant offers support to our HIV researchers by providing the information they seek from the data and potentially generating SAS, R, or Python code for desired variable names. For instance, if we require a dataset containing HIV PrEP information, we can request it to generate a list of current datasets that contain data on PrEP medication usage in 2019, along with the associated variable names. Subsequently, we can instruct it to produce lines of code, such as a logistic regression, which can be readily copied and pasted into our analysis software of choice. Another example would be if our researchers would like information returned about a given dataset, this assistant could return specific information based on the researcher's query.",Acquisition and/or Development,Neither,11/1/2024,11/1/2024, , ,Developed in-house., , , , , , , , ,Yes,,"Internal documentation containing mapped eHARS LOINC codes, IQVIA data dictionaries",Documentation is complete,,, , , ,Yes,Enterprise Data Analytics and Visualization (EDAV) Platform,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Identify infrastructure supports for physical activity (e.g. sidewalks) in satellite and roadway images,Department of Health and Human Services,HHS,CDC,Health & Medical,,None of the above.,"The physical environment, such as sidewalk availability, in areas may be correlated to levels of physical activity. The Physical Activity (PA) and Health Branch is responsible for monitoring and gathering data on physical activity behaviors, as well as assessing the environments that promote physical activity. The implementation of this technology has the potential to streamline the surveillance of sidewalks, which play a crucial role in supporting physical activity (PA). The existing methods for cataloging sidewalks, such as manual inspection, are labor-intensive and costly. However, this technology has the capability to significantly minimize the effort required for such tasks.","The Division of Nutrition, Physical Activity, and Obesity within CDC's National Center for Chronic Disease Prevention and Health Promotion aims to explore and advance the use of machine learning methods for identifying sidewalks, bicycle lanes, and other relevant infrastructure in various types of images, including satellite and roadway images. The input data would consist of image-based data, while the outputs could take the form of geocoded data tables, maps, GIS layers, or summary reports. The goal is to develop and promote these techniques to enhance understanding and analysis of built environments related to physical activity.",Acquisition and/or Development,Neither,9/1/2022,9/1/2023, , ,Developed with both contracting and in-house resources.,N/A, , , , , , , ,No,,none - we used publicly-available images,Documentation is complete,,, , , ,No, ,Less than 6 months,Yes,,Yes,,No,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Immunization Information Systems Guidance Documentation Navigation and Management (IDAB EDAV Azure OpenAI Technology Use),Department of Health and Human Services,HHS,CDC,Mission-Enabling (internal agency support),,None of the above.,"This AI solution (chatbot) helps with retrieving information and updating guidance documentation by increasing interactivity and discoverability of information in the Immunization Information Systems (IIS) domains of HL7 electronic data exchange standards, CDSi (Clinical decision support for immunizations), and IIS operational best practices. It provides a modernized approach to knowledge retrieval and real-time guidance, as well as support the creation of new guidance by leveraging the AI capabilities in natural language processing and content generation to draft, refine, and validate new guidance documents efficiently and accurately. This AI solution creates a better, faster, and more actionable method for IIS SMEs to obtain IIS guidance from previously published and publicly available IIS documentation. This solution will provide a more modernized approach to knowledge retrieval and real-time guidance, as well as support the creation of new guidance by leveraging the chatbots capabilities in natural language processing and content generation to draft, refine, and validate new guidance documents efficiently and accurately. This AI solution (chatbot) provides an efficient and user-friendly way to search, interact, and update these IIS guidance documents. It can help experts answer questions faster, new employees find information more easily, and employees better understand best practices.","This AI solution (chatbot) uses Retrieval Augmented Generation (RAG) to retrieve and synthesize information from a limited set of publicly available IIS guidance documents related to the HL7 Electronic Exchange Standard specification, Clinical Decision Support for Immunization (CDSi) and IIS Operational Best Practices. It does this using a Question-and-Answer style interface, allowing team members to retrieve answers in an efficient and user-friendly way. On 07/26/2024 the CDC Information Technology and Data Governance (ITDG) Executive Committee has approved this investment “to use AI to create a system that can conduct a dialogue with a user and answer questions about the contents of already approved public documents”.",Acquisition and/or Development,Neither,10/1/2022,8/1/2024, , ,Developed with both contracting and in-house resources.,"75D301-22-C-13291, 75D30124C20195", , , , , , , ,No,,"We did not do any training or fine tuning, but we used the following materials to evaluate the chatbot: HL7 Documentation Data, CDSi Logic Specification 4.4 Data. This information is entirely publicly accessible.",Documentation is widely available,,, , , ,Yes,Enterprise Data Analytics and Visualization (EDAV) Platform,Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, LaserAI,Department of Health and Human Services,HHS,CDC,Health & Medical,,None of the above.,"This project aims to evaluate a commercial off the shelf, software as a service systematic review project management software. This software uses machine learning/ natural language processing to support title and abstract screening, PDF retrieval, full text review, and most importantly data extraction to reduce screening time and improve accuracy of all phases of a systematic review. The extracted data will be synthesized and graded, and will be used to inform the development of evidence-based infection prevention and control recommendations for healthcare settings.","This AI uses natural language process to (1) identify potentially relevant articles and prioritize them for screening, (2) retrieve PDFs from PubMed, and (3) suggest data in PDFs for extraction.",Acquisition and/or Development,Neither,4/1/2024,4/1/2024, , ,Developed with contracting resources.,N/A, , , , , , , ,No,,"None: Any data used to train the AI is publicly available, peer-reviewed data.",Documentation is missing or not available,,, , , ,No, ,Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, NewsScape,Department of Health and Human Services,HHS,CDC,Mission-Enabling (internal agency support),,None of the above.,"Development of early warning indicators. News articles can form an early warning indicator of public health events and other pieces of information which can be utilized across all major domains of public health. Because of the quantity of news articles, manual efforts are impossible to gather this information. There are a variety of endpoints that this AI output could help support. Various teams across the CDC have expressed interest in being able to quickly get the right news content, with summaries of those articles to help support outbreak detection, report generation, surveillance and monitoring of pathogen specific news, etc. AI lets us efficiently filter and summarize thousands of news articles a day into a handful of daily ""news events"" that user can glean information from. ","NewsScape is a AI enabled news aggregation and summarization that is hosted within the 1CDP platform. The main motivation to build  NewsScape was to develop a system that uses Large Language Models (LLMs) to surface relevant insight from recent news articles. NewsScape ingests a high volume of news articles, in the order of thousands of news articles everyday, and surfaces the information from them that related to the topic we were interested (for example, pathogen related news articles, or U.S. medical supply chain updates).  The tool can be customized based on specific program office needs, and can be deployed independently of one another so that one program office can have their own custom version of NewsScape installed.",Acquisition and/or Development,Neither,1/1/2023,1/1/2023, , ,Developed with both contracting and in-house resources.,N/A, , , , , , , ,No,,None,Documentation has been partially completed,,, , , ,Yes,1 CDC Data Platform (1CDP),6-12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, RAPID Analysis of Policy and Program Documents (RAPID),Department of Health and Human Services,HHS,CDC,Mission-Enabling (internal agency support),,None of the above.,"Policy and documentation awareness and evaluation The web application will: 1) save staff time by streamlining laborious document reviews; 2) expand capacity by reducing the need for specialized staff training and skills; 3) provide complete, plain language answers to policy surveillance and evaluation questions along with binary codes or scores; 4) answer questions consistently (i.e., reduce intra-rater variability for redundant coding); and 5) enable easy validation of AI-provided answers.",An internal web application will enable CDC users to: 1) import and securely store policy or program documents; 2) search for relevant documents; 3) ask questions of relevant text segments; 4) validate answers; and 5) collaborate on policy surveillance and evaluation projects.,Acquisition and/or Development,Neither,11/1/2021,9/1/2022, , ,Developed with contracting resources.,N/A, , , , , , , ,No,,"RAPID analysis of DNPAO policy and program data using GPT will results in project specific databases. To ensure accuracy and reliability of these AI-generated data, they will be compared to data of CDC manual reviews of the same documents by SMEs. ",Documentation is missing or not available,,, , , ,Yes,Enterprise Data Analytics and Visualization (EDAV) Platform,Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, SewerScout: Automated on-site sewage facility detection from aerial imagery to identify failed systems,Department of Health and Human Services,HHS,CDC,Health & Medical,,None of the above.,"This business process focuses on the automation of the accurate identification of onsite wastewater systems in state, tribal, local, and territorial (STLT) health jurisdictions. This application scans aerial imagery and uses object detection and image classification models to detect onsite sewage facilities. Initial work is focusing on the ability to identify known systems. The intent of this project is to identify failed systems, which can be sources of soil and drinking water contamination as well as vectors that carry or transmit disease in the community. This project will allow STLT partners to readily characterize the location and functional status of septic systems without the time and resource consuming effort of visiting each household. The resources in many local health districts (LHDs) are barely adequate to undertake assessments of suitability for new systems (if not entirely contracted out) and most LHDs do not have any program to routinely inspect current systems. If successful, this project will allow state, tribal, local, and territorial public health departments to more easily identify failing septic systems and work to address them. If successful, subsequent work will work on differentiating functional from failed systems. Identification of septic systems during routine operations would help bolster and expedite disaster response efforts after major disasters by having a ready catalog of systems and using pre- / post-disaster satellite images to direct resources in a much more efficient way than on-the-ground door-to-door surveys in rural / remote locations. ","The intent of this project is to identify failed wastewater systems, which can be sources of soil and drinking water contamination as well as vectors that carry or transmit disease in the community.",Acquisition and/or Development,Neither,1/1/2023,1/1/2023, , ,Developed in-house., , , , , , , , ,No,,The project has not selected the final sources of data but no agency-owned data is anticipated to be used to train the model.,Documentation is widely available,,, , , ,Yes,Enterprise Data Analytics and Visualization (EDAV) Platform,More than 12 months,Yes,,Yes,,No,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Sidekick Comms bot Offering User-friendly Tips (SCOUT),Department of Health and Human Services,HHS,CDC,Mission-Enabling (internal agency support),,None of the above.,"The AI will impact the content creation processes for CDC.GOV and reduce burden in creating new webpages. CDC sometimes uses GenAI (generative artificial intelligence) to help create content for the public. This may include content for the web, social media, fact sheets, and graphics. In every case, our experts review our communication materials in an effort to ensure accuracy and quality before use. CDC public health research, data, and recommendations are based on the best science currently available and go through a separate agency review process prior to release. The solution aims to reduce the workload of health communicators by simplifying the process of creating new web pages and social media posts. By leveraging Generative AI technology, we can produce new content much more quickly than if communications staff were to do it on their own. The goal is to make the published content more accessible and understandable for the general public, ensuring that CDC information reaches a wider audience. It's important to note that all content generated by the AI is carefully reviewed and edited by a human before it is used and the follow disclaimer is stated with public content that this tool has been used with: ""CDC sometimes uses GenAI (generative artificial intelligence) to help create content for the public. This may include content for the web, social media, fact sheets, and graphics. In every case, our experts review our communication materials in an effort to ensure accuracy and quality before use. CDC public health research, data, and recommendations are based on the best science currently available and go through a separate agency review process prior to release.""","CDC's National Center for Chronic Disease Prevention and Health Promotion use of Generative AI in this use case is to help convert existing web content into plain and clear language. This solution uses Gen AI to create new versions of publicly available content that are easier for end-users to understand. By incorporating prompts, the tool makes the content more suitable for a general lay audience. In addition to web articles, the Generative AI tool also allows for the creation of social media posts. The goal of this project is to make information more accessible by providing plain language content on various platforms. CDC sometimes uses GenAI (generative artificial intelligence) to help create content for the public. This may include content for the web, social media, fact sheets, and graphics. In every case, our experts review our communication materials in an effort to ensure accuracy and quality before use. CDC public health research, data, and recommendations are based on the best science currently available and go through a separate agency review process prior to release.",Acquisition and/or Development,Neither,12/1/2024,1/1/2024, , ,Developed with contracting resources.,NCCDPHP OD IAA, , , , , , , ,Yes,,None,Documentation is complete,,, , , ,Yes,Enterprise Data Analytics and Visualization (EDAV) Platform,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Transcribing Cognitive Interviews with Whisper,Department of Health and Human Services,HHS,CDC,Mission-Enabling (internal agency support),,None of the above.,"Federal Health Survey Research. Specifically, Cognitive interviews are a form of qualitative research where interviewers ask probing questions to understand how the interviewee understands a concept which is being asked about in a federal survey. These interviews are a key part of survey design to identify any potential misconceptions and prevent lengthy and hard to answer questions which could create potentially wrong health statistics. These interviews are time consuming to perform and report as they are inherently individual and require extensive hours to transcribe. By using AI to transcribe the interviews, researchers can potentially cut down on the hours or qualitative review, increasing the speed of research publication and quality of survey questions used on federal surveys. Without transcripts, these interviews have to be listened to repeatedly taking at least the same amount of time as the full length of the interview to find important pieces of information for their work. These transcripts can greatly reduce the burden on researchers by enabling immediate comparison of concepts and answers along with timestamps created for researchers to reference in review.",This AI generates transcripts from recorded interviews for staff to use when performing qualitative research to assist with Federal Health Survey research.,Acquisition and/or Development,Neither,10/1/2022,7/1/2024, , ,Developed in-house., , , , , , , , ,No,,"None, publicly available data was used to evaluate performance",Documentation has been partially completed,,, , , ,No, ,More than 12 months,Yes,,Yes,,No,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Use of Natural Language Processing for Topic Modeling to Automate Review of Public Comments to Notice of Proposed Rulemaking,Department of Health and Human Services,HHS,CDC,Mission-Enabling (internal agency support),,None of the above.,"This use case is aimed to assist in the review of public comments for Notices of Proposed Rulemaking. All responses are manually reviewed following this tool usage. By law, Public comments are required to be manually review for any notice of public rule making. These comments are highly labor intensive due to both the quantity and wide range of potential opinions shared by the public. These comments can range in both length of response and quantity serving as a challenge for properly ensuring the insights from the public are accurately gathered in the review process. By using an LLM based Topic Modeling approach, manual reviewers can organize responses based upon theme to enhance both the insights gained, the speed of manual review, and ensure all topics mentioned can be accurately reported in all review processes. This process can use open source python libraries and open-source LLMs to complete different actions. This AI methodology enables manual review processes to be more effective in gaining insights from the public, better organized in themes and categories present within text, and reduce the burden of manual review of comments from notices of proposed rulemaking required by law.",Generates clusters of similar public comments to a 'notice of proposed rulemaking' to aid in manual review.,Acquisition and/or Development,Neither,2/1/2023,4/1/2023, , ,Developed in-house., , , , , , , , ,No,,None,Documentation has been partially completed,,, , , ,Yes,"Enterprise Data, Analytics, and Visualization Platform",Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Center for Forecasting and Outbreak Analytics Disease Modeling,Department of Health and Human Services,HHS,CDC,Health & Medical,,None of the above.,"Federal and State, Tribal, Local, and Territorial health department responses to disease outbreaks depends on being able to predict the future behavior of a given outbreak as well as the potential impact of different responses to the outbreak. This project aims to build models of disease spread that allow for (1) determining if and where infection/impact rates are growing or increasing (2) forecasting future disease progression (3) modeling specific scenarios such as the impact of vaccination. We use model output to provide information to Federal and SLTT decision makers to improve disease response. This allows those decision makers to determine where and how to direct resources in a disease out break as well as the impact of potential response actions. This kind of insight is critically dependent on mathematical modeling.","This project models disease spread and impact based upon a set of models, a variety of data sources, and allows for the inclusion of the impact of potential public health actions such as vaccinations. These insights can be used by public health decision makers to inform current and future responses to public health emergencies",Implementation and Assessment,Neither,4/1/2022,4/1/2022, , ,Developed with both contracting and in-house resources.,75D30123F16628, , , ,No, ,No,No,Yes,,"A wide array of public health data including hospital admissions data from NHSN, emergency department visit data from NSSP, case reports provided to CDC from local jurisdictions, variant prevalence data sourced from public health laboratory samples, wastewater data from NWSS, and other related sources.",Documentation is complete,,,Yes,Yes – source code is publicly available.,"https://github.com/cdcgov/dynode; https://github.com/cdcgov/ixa -- note, not all code is currently public",Yes,"Enterprise Data Analytics and Visualization (EDAV) Platform, EDAV-EXT, 1 CDC Data Platform (1CDP)",Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, ChatCDC- CDC Enterprise GenAI Chatbot: Bring your own Enterprise Data (Data Analysis),Department of Health and Human Services,HHS,CDC,Mission-Enabling (internal agency support),,None of the above.,"This AI system is an extension of the GenAI chatbot but is only available to a small group of pilot testers as of November 5, 2024. It operates through an ATO'ed instance of Microsoft Azure OpenAI through CDC's Enterprise Data Analytics and Visualization Platform. Using a method called retrieval augmented generation (RAG), this enhancement enables staff to augment the existing GenAI chatbot with access and “knowledge” of internal or enterprise data such as documents, spreadsheets, and PDFs. Additionally, each chatbot response has citations referencing to the sections of source text that informed the response, which facilitates human-in-the-loop validation for accuracy and relevancy of AI-generated content. One way users are using this tool is for data extraction, which accelerates analysis of large document sets. By extracting key points from large volumes of text in various file formats, the enterprise-data-enhanced chatbot streamlines access to critical insights from reports, internal documentation, publications, and other documents. For example, one team is using the chatbot to support literature review by extracting information related to housing status and HIV in the US from a set of pre-screened publications. Examples of this information include a binary yes/no on whether the research paper focuses on the topic of interest, the paper’s population of focus, and the type of paper. The enterprise data chatbot’s ability to quickly and accurately extract relevant information reduces manual search time and errors associated with human data retrieval. The output will be leveraged for more accurate data analysis, timely insights, and better-informed decision-making processes.","When the user enters a prompt or query related to data extraction into the chatbot, the tool uses a GPT-4 model and Azure AI Search within a RAG architecture to retrieve relevant sections of text from the provided enterprise data sources and extract key information as instructed by the user’s prompt.",Implementation and Assessment,Neither,6/1/2024,6/1/2024, , ,Developed with both contracting and in-house resources.,N/A, , , ,No, ,No,No,No,,None,Documentation is missing or not available,,,Yes,"Yes – agency has access to source code, but it is not public.", ,Yes,Enterprise Data Analytics and Visualization (EDAV) Platform,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, ChatCDC- CDC Enterprise GenAI Chatbot: Bring your own Enterprise Data (Summarization),Department of Health and Human Services,HHS,CDC,Mission-Enabling (internal agency support),,None of the above.,"This AI system is an extension of the GenAI chatbot but is only available to a small group of pilot testers as of November 5, 2024. It operates through an ATO'ed instance of Microsoft Azure OpenAI through CDC's Enterprise Data Analytics and Visualization Platform. Using a method called retrieval augmented generation (RAG), this enhancement enables staff to augment the existing GenAI chatbot with access and “knowledge” of internal or enterprise data such as documents, spreadsheets, and PDFs. Additionally, each chatbot response has citations referencing to the sections of source text that informed the response, which facilitates human-in-the-loop validation for accuracy and relevancy of AI-generated content. The enterprise data chatbot’s summaries will enhance operational, informational, and decision-making processes that would otherwise require extensive research and reading time. For an example of a more informational use case, one team provided the chatbot with documentation and resources about a division’s organizational structure, projects, administrative processes, and tools to support the orientation of new staff and (re)orientation of existing staff. The chatbot enables the division’s staff to efficiently find resources for their roles; better understand the division’s initiatives; and track the activities involved in the onboarding process for new staff.","When the user enters a prompt or query related to summarization into the chatbot, the tool uses a GPT-4 model and Azure AI Search within a RAG architecture to retrieve relevant sections of text from the provided enterprise data sources and condense that information into concise summaries in response to the user’s prompt. ",Implementation and Assessment,Neither,6/1/2024,6/1/2024, , ,Developed with both contracting and in-house resources.,N/A, , , ,No, ,No,No,No,,None,Documentation is missing or not available,,,Yes,"Yes – agency has access to source code, but it is not public.", ,Yes,Enterprise Data Analytics and Visualization (EDAV) Platform,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, Leveraging AI for Metadata Tagging for Enterprise Data Catalog of CDC,Department of Health and Human Services,HHS,CDC,Mission-Enabling (internal agency support),,None of the above.,"CDC’s Enterprise Data Catalog provides an internal single centralized inventory of available datasets within CDC staff. It provides data management and search tools to help users find what they need and aims to provide descriptions to aid in relevance determinations for users. This is from the appropriate metadata, e.g. project title, description, or other important information describing the dataset. Because of the previous manual metadata tagging process, many datasets catalogued do not have the appropriate tags to help staff determine the relevance of any particular dataset available within CDC for particular tasks. Because of the lack of standardization and gaps in metadata fields, this AI's output will greatly increase the speed for applying the appropriate metadata fields for each dataset within the catalogue to increase the usability of the product. This reduces the time and effort required for manual tagging and improves the consistency and completeness of metadata. ",AI is used in this case to generate suggested metadata fields based upon existing metadata information provided by a dataset. These tags would then be used for staff accessing the enterprise data catalogue to discover datasets and find connections to support existing on-going work.,Implementation and Assessment,Neither,6/1/2024,6/1/2024, , ,Developed in-house., , , , ,No, ,No,No,Yes,,Enterprise Data Catalog Metadata fields,Documentation has been partially completed,,,Yes,"Yes – agency has access to source code, but it is not public.", ,Yes,Enterprise Data Analytics and Visualization (EDAV) Platform,Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Malaria parasites DNA barcode geography classification,Department of Health and Human Services,HHS,CDC,Health & Medical,,None of the above.,"Epidemiological investigations of Malaria To complement epidemiologic investigations of domestic malaria cases, including cases that were domestically acquired. This information can help assess the origin of the malaria strain in question, which can help us understand how the strain entered the US. For more information, please see the following manuscript: https://journals.asm.org/doi/full/10.1128/aac.01203-24","Algorithm examines a sequence barcode/genotype (our use case is for malaria, but it can be applied to other contexts) and it assigns the malaria parasite genotype to a geographic origin, which may be a continent or some other geographic regional category. While this use case has the barcode as a set of genetic loci from malaria parasites, and the categories are geographic (i.e., continents, subregions), the algorithm could be trained to assign genetic barcodes from other pathogens to any number of different categories assuming the appropriate training dataset exists.",Implementation and Assessment,Neither,7/1/2023,7/1/2023, , ,Developed in-house., , , , ,No, ,No,No,No,,"Data used are a mixture of data generated at CDC and other data available publicly. All CDC data are accessible via the following links: https://www.ncbi.nlm.nih.gov/bioproject/PRJNA428490/, https://www.ncbi.nlm.nih.gov/bioproject/PRJNA1092573/, https://www.ncbi.nlm.nih.gov/bioproject/?term=PRJNA1110244. Travel histories from case patients were used to assess performance of the model to compare the generated output of the model and are available in the following manuscript: https://journals.asm.org/doi/full/10.1128/aac.01203-24. Non-CDC owned data are available through this link: https://apps.malariagen.net/apps/pf7/",Documentation is widely available,,,Yes,"Yes – agency has access to source code, but it is not public.", ,Yes,Advanced Molecular Detection Scientific Computing Platform,6-12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, The School Closure Awareness System,Department of Health and Human Services,HHS,CDC,Emergency Management,,None of the above.,"Identifying unplanned school closures. CDC is responsible for supporting schools before, during, and after and emergency. During COVID 19 this included monitoring school closure or virtual status and reporting this to federal partners, the White House, and the public. Due to cost the process used during COVID was no longer feasible and ceased in DEC 2022. We explored multiple ways to capture this data and our current system met the need best. This AI has allowed us to save nearly 2-million-dollars from contracting fees and decrease human work hours by 200 hours. It has made the capture of some of this data possible after cessation of the process used during COVID 19 which we could no longer support. The AI supported system is faster and captures more information about individual closures than the original human-powered process.","This system uses publicly available Facebook posts from approximately 40,000 school or district Facebook accounts. Then uses a large language model to categorize posts as unplanned school closure for several types of events (weather, health, facility issues, safety). It also will denote the type of status change (full closure, virtual, hybrid, or early / late dismissal). The data is checked and recoded (if needed) by a staff person every 24 hours.",Implementation and Assessment,Neither,7/1/2022,11/1/2022, , ,Developed in-house., , , , ,No, ,No,No,No,,None,Documentation has been partially completed,,,Yes,"Yes – agency has access to source code, but it is not public.", ,Yes,"Enterprise Data, Analytics, and Visualization Platform",More than 12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, Using Generative AI for Stance Analysis of Public Comments on CDC’s Proposed Rules,Department of Health and Human Services,HHS,CDC,Mission-Enabling (internal agency support),,None of the above.,"By law, Public comments are required to be manually review for any notice of public rule making. These comments are highly labor intensive due to both the quantity and wide range of potential opinions shared by the public. These comments can range in both length of response and quantity serving as a challenge for properly ensuring the insights from the public are accurately gathered in the review process. By using an LLM based Topic Modeling approach, manual reviewers can organize responses based upon theme to enhance both the insights gained, the speed of manual review, and ensure all topics mentioned can be accurately reported in all review processes. This process can use open source python libraries and open-source LLMs to complete different actions. As public feedback on various rules gets collected across the CDC’s divisions, this initiative along with other NLP approaches can potentially save time for CDC’s public policy experts during public comment review. ","Reviewing the public comments currently involves a manual review process that necessitates a high level of time and effort on the part of regulatory analysts. To improve the process of reviewing public comments as part of regulatory analysis, our team implemented topic modeling (Grootendorst et al., 2022) and sentiment analysis (Hartman et al., 2023) of comments published to the new amendment to foreign quarantine regulation published by CDC on July 10, 2023. This solution expands these efforts by using generative AI models to try for capture stance expressed in each comment. While there is also an option to manually label and train models for stance detection, custom-trained text classification models may not be transferable to future rules due to rule-specific discussions and arguments. In contrast, using generative AI models for stance detection can be transferable to future rules irrespective of the discussion contexts and arguments.",Implementation and Assessment,Neither,7/1/2023,7/1/2023, , ,Developed in-house., , , , ,No, ,No,No,No,,NA,Documentation is missing or not available,,,Yes,"Yes – agency has access to source code, but it is not public.", ,No, ,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, A reusable NLP pipeline for clinical narratives preprocessing and characterization,Department of Health and Human Services,HHS,CDC,Health & Medical,,None of the above.,"Data cleaning of Clinical narratives from health records. At scale, clinical narratives are a dataset available to CDC currently underutilized as there is not a structured pipeline to identify structured information output. We are currently working on setting up a process to create a pipeline that cleans data markup (html, rtf, xml) and identifies signs and symptoms using pre-trained Named Entity Recognition (NER) tasks from relevant models published in peer review journals. The use of these pretrained models significantly reduces the amount of time a human to spend tagging data and provides additional features and increased consistency from unstructured data for identification of more novel insights. ","The objective of this project is to take unstructured data, clean it, and use pretrained Named Entity Recognition (NER) tools to extract signs/symptoms in deidentified electronic health records that can be used to characterize a cohort or make further analysis.",Initiated,Neither,7/1/2024,, , , , , , , , , , , , ,, , ,,, , , , , ,, ,, ,,,, , ,,,,,,,,,,,,,,,,,, Autocoding to Support Adverse Drug Event Surveillance,Department of Health and Human Services,HHS,CDC,Mission-Enabling (internal agency support),,None of the above.,"The model will help surveillance epidemiologists working at CDC to determine whether reported potential adverse drug events in the National Electronic Injury Surveillance System Cooperative Adverse Drug event Surveillance meet the agency's surveillance case definition for the same. It will speed up coding of the drug event reports, helping epidemiologists produce prevalence estimates for the surveillance system.","The model will take a de-identified free-text description of the patient's emergency department visit encounter, along with a handful of other pre-coded variables, and output the probability that the encounter meets the case definition.",Initiated,Neither,5/1/2024,, , , , , , , , , , , , ,, , ,,, , , , , ,, ,, ,,,, , ,,,,,,,,,,,,,,,,,, Automating LIMS Bioinformatics Workflow Configuration and Enhancing Lab Quality Management with AI,Department of Health and Human Services,HHS,CDC,Mission-Enabling (internal agency support),,None of the above.,"The tool proposed will impact the laboratory bioinformatics workflow configuration within the Core Laboratory's Genomics Sequencing Lab (GSL). Specifically, it will improve the efficacy with Clarity LIMS for tracking and processing Next Generation Sequencing (NGS) samples. This includes the creation and customization of bioinformatics workflows in the Laboratory Information Management System, which is currently a time-consuming and expertise-intensive task for bioinformaticians and laboratory scientists. The RAG system can potentially reduce the time required to configure Clarity LIMS workflows, enabling rapid deployment which is particularly crucial during pandemics or outbreak responses. By simplifying the workflow customization process through natural language inputs, it lowers the barrier to entry for laboratory scientists and bioinformaticians who may not be familiar with XML scripting. The system also enhances team learning and training by providing easy access to relevant documents on quality management and regulations, ensuring that all team members are well-informed about best practices.","We propose to use a Retrieval-Augmented Generation (RAG) framework for two primary functions: It converts natural language lab protocols into precise XML workflows that are compatible with Clarity LIMS. This conversion process involves understanding technical documentations and generating corresponding script code. In addition, it serves as an interactive knowledge base for documentation of Clarity LIMS and Laboratory Quality Management and Regulations. ",Initiated,Neither,1/1/2024,, , , , , , , , , , , , ,, , ,,, , , , , ,, ,, ,,,, , ,,,,,,,,,,,,,,,,,, DGMH AI Chatbot,Department of Health and Human Services,HHS,CDC,Mission-Enabling (internal agency support),,None of the above.,"The DGMH AI chatbot would assist with developing an initial draft response to inquiries where the response could have been found on our website. This would allow DGMH staff to have a starting point on a response to edit and clear. Additionally, the response could be adjusted for plain language to assist in ensuring the inquirer understands the response being sent back to them. The AI chatbot would allow inquiries to be responded to quicker and would allow staff to focus on other high priority items as well. The project will rely on using Retrieval-Augmentation Generation (RAG) to identify relevant information from CDC’s relevant website pages to generate relevant responses using AI. Evaluation of the project will assess how accurate the response was, how complete it was, and how much revision was needed. Evaluation would also explore the answer and if the answer that was generated was consistent throughout time if multiple people asked similar questions. The goal is consistency of content rather than tone/plain language. The chatbot will help to reduce the turnaround time for responding to inquiries and free up time for staff to address other priorities.","CDC's Division of Global Migration Health receives inquires for information which is available publicly in some form but may be hard to find and requires staff responding to these inquiries to have a broad knowledge and can be time intensive to respond to. The AI Chatbot will draft an initial response to questions using content available on CDC’s public-facing webpages. Each response goes through the existing CDC clearance process regardless of if AI is used, but the Chatbot helps provide an initial draft based on previously published content. The goal is to help with responding to different inquiries the division receives, including but not limited to CDC-INFO and media. ",Initiated,Neither,10/1/2024,, , , , , , , , , , , , ,, , ,,, , , , , ,, ,, ,,,, , ,,,,,,,,,,,,,,,,,, Distiller SR: AI to screen research articles for Community Guide reviews,Department of Health and Human Services,HHS,CDC,Mission-Enabling (internal agency support),,None of the above.,"This tool will be used to screen research articles that will be used to inform Community Preventive Services Task Force recommendations (a body that produces guidelines independent of CDC). Use of this technology may increase the speed with which systematic reviews can be conducted. This may expedite the time it takes to evaluate the effectiveness of public health programs for a CPSTF recommendation (not a CDC guideline). ",Machine learning will be used to more efficiently screen research articles relevant to evaluating the effectiveness of an intervention. ,Initiated,Neither,5/1/2023,, , , , , , , , , , , , ,, , ,,, , , , , ,, ,, ,,,, , ,,,,,,,,,,,,,,,,,, Evaluating Generative AI for polio containment.,Department of Health and Human Services,HHS,CDC,Health & Medical,,None of the above.,"The AI will impact the data retrieval and automation process for HIV researchers, specifically in providing information from datasets and potentially generating code for desired variable names and analysis or generating information about a particular dataset. As polio eradication progresses, containment of poliovirus becomes critical. This project aims to identify and evaluate Generative AI tools to extract information from publications in an effort to identify poliovirus in places previously thought to be polio-free. Identification of such a tool may diversify Polio and Picornavirus Branch's surveillance portfolio and increase Branch's capability of processing large volume of data for containment purposes. This project may support CDC PPLB’s polio surveillance activity and containment efforts.","This AI assistant will use retrieval augmented generation (RAG) to return information related to HIV based on a user's query. Our aim is to input all dataset documentation and metadata into an index with the goal of creating a virtual assistant. This assistant offers support to our HIV researchers by providing the information they seek from the data and potentially generating SAS, R, or Python code for desired variable names. For instance, if we require a dataset containing HIV PrEP information, we can request it to generate a list of current datasets that contain data on PrEP medication usage in 2019, along with the associated variable names. Subsequently, we can instruct it to produce lines of code, such as a logistic regression, which can be readily copied and pasted into our analysis software of choice. Another example would be if our researchers would like information returned about a given dataset, this assistant could return specific information based on the researcher's query.",Initiated,Neither,11/1/2023,, , , , , , , , , , , , ,, , ,,, , , , , ,, ,, ,,,, , ,,,,,,,,,,,,,,,,,, Respiratory Virus Response (RVR) Data Analysis Concept,Department of Health and Human Services,HHS,CDC,Health & Medical,,None of the above.,"This solution will impact the process of summarizing subject matter expert (SME) interpretations/knowledge during a response. Response efforts require prompt turnaround of essential information to the public. This project will impact public health by improving the efficiency of dissemination of information through production of key takeaways which can then be quickly reviewed by SMEs and cleared. This project will also allow for improved understanding of limitations and uncertainties in use of AI methods that could contribute to improved implementation of such methods in public health.","The goal of this solution is to evaluate how AI can be implemented in the process of summarizing subject matter expert (SME) interpretations/knowledge to improve efficiency in knowledge dissemination during a response. As part of the proof-of-concept, we seek to evaluate the limitations and uncertainties of AI methods in this use-case, such as bias and hallucinations as such issues will require mitigation before this method can be broadly adopted. Our proposed approach would make use of the ChatGPT interface through EDAV (or alternative accepted platform) as our prompts will include uncleared data from the Respiratory Virus Response (RVR). In this response, SMEs from different groups provide bulleted information that then needs to be further summarized to address key takeaways. Bullets from prior weeks will prompt AI to generate summaries. The initial approach will focus on engineering prompts to get key takeaways while future iterations may require fine-tuning to allow for better contextualization of responses as well as improvements in tone and style. ",Initiated,Neither,6/1/2024,, , , , , , , , , , , , ,, , ,,, , , , , ,, ,, ,,,, , ,,,,,,,,,,,,,,,,,, School LLM initial abstract review process ,Department of Health and Human Services,HHS,CDC,Mission-Enabling (internal agency support),,None of the above.,"Reviewing research related to school readiness science. CDC/DRRS' School Preparedness unit is responsible for developing research around readiness science. As part of this a large language model was used to abstract themes and code publications that were identified as related to school closure to aid us in supporting our research. If funding were to become available, we would like to develop this static, one-time product into a modifiable tool that could be used for ongoing article abstraction across school and other settings and population. The AI allows us to efficiently categorize thousands of abstracts in a much shorter time-frame with much less human time required into a user-friendly dashboard. ",The AI used a LLM to extract data from an abstract review and categorize relevant themes and topics into a user-friendly dashboard. This dashboard can be used by a health scientist to pull resources from 2012-2022 for specific school closure outcomes or themes.,Initiated,Neither,8/1/2023,, , , , , , , , , , , , ,, , ,,, , , , , ,, ,, ,,,, , ,,,,,,,,,,,,,,,,,, ChatCDC - CDC Enterprise Generative AI Chatbot (Content Editing),Department of Health and Human Services,HHS,CDC,Mission-Enabling (internal agency support),,None of the above.,"CDC has an internal productivity tool available to all CDC staff which can be used in a variety of ways. This use case covers staff who will be using this general purpose to assist with editing text previously written, or content editing. This may refer to using the chatbot to help refine emails or messages to other CDC staff, reviewing internal documents, reviewing scheduling or other bureaucratic information and other such information. CDC sometimes uses GenAI (generative artificial intelligence) to help create content for the public. This may include content for the web, social media, fact sheets, and graphics. In every case, our experts review our communication materials in an effort to ensure accuracy and quality before use. CDC public health research, data, and recommendations are based on the best science currently available and go through a separate agency review process prior to release. Currently, CDC has internal Generative AI guidance determining how this tool can be used, and individual Centers, institutes, or Offices may implement their own policy to ensure safe and responsible AI. This includes internal guidance on using AI for External Communications. This tool enables CDC staff to have the productivity benefits of Generative AI powered chatbots without any of the security risks of data uploads. This tool can be used by all CDC staff to support general business operations and enables a quick review for finding any spelling or grammatical errors, fixing formatting, or other copy-editing type corrections and saves staff time will increasing the quality of communications. CDC sometimes uses GenAI (generative artificial intelligence) to help create content for the public. This may include content for the web, social media, fact sheets, and graphics. In every case, our experts review our communication materials in an effort to ensure accuracy and quality before use. CDC public health research, data, and recommendations are based on the best science currently available and go through a separate agency review process prior to release.","This chatbot, powered by Azure OpenAI Large Language Models, will generate responses to questions asked by the user. In this case, responses will be recommendations on edits for content provided by the user such as a draft email. The chatbot's output can then be copied and pasted or manually entered based upon the users review. CDC sometimes uses GenAI (generative artificial intelligence) to help create content for the public. This may include content for the web, social media, fact sheets, and graphics. In every case, our experts review our communication materials in an effort to ensure accuracy and quality before use. CDC public health research, data, and recommendations are based on the best science currently available and go through a separate agency review process prior to release.",Operation and Maintenance,Neither,10/1/2023,11/1/2023,2/1/2024, ,Developed with both contracting and in-house resources.,N/A, , , ,No, ,No,No,No,,Yes. A series of pilot projects was conducted to evaluate the performance of this chatbot and these pilot projects are included within this inventory.,Documentation is widely available,,,Yes,"Yes – agency has access to source code, but it is not public.", ,Yes,Enterprise Data Analytics and Visualization (EDAV) Platform,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, ChatCDC - CDC Enterprise Generative AI Chatbot (Data Extraction),Department of Health and Human Services,HHS,CDC,Mission-Enabling (internal agency support),,None of the above.,"CDC has an internal productivity tool available to all CDC staff which can be used in a variety of ways. This use case covers staff who will be using this general purpose tool to assist with extracting information from larger pieces of text. Currently, CDC has internal Generative AI guidance determining how this tool can be used, and individual Centers, institutes, or Offices may implement their own policy to ensure safe and responsible AI. From larger documents which may be time consuming to find specific pieces of information, this AI system enables staff to extract key pieces of information to be used in later tasks. This can include action items from meeting transcripts, specific discussion focuses from larger meetings, specific mentions, or other such information. this makes the finding of information from larger document far faster and reduces the manual burden of reading larger documents for one or two specific pieces of information.","This chatbot, powered by Azure OpenAI Large Language Models, will generate responses to questions asked by the user, in this case about some larger piece of text and any specific pieces of information to extract. The chatbot's output can then be copied and pasted or manually entered into the appropriate testing process by the staff to ensure any code used from the chatbot's output meets the criteria of the current task.",Operation and Maintenance,Neither,10/1/2023,11/1/2023,2/1/2024, ,Developed with both contracting and in-house resources.,N/A, , , ,No, ,No,No,No,,Yes. A series of pilot projects was conducted to evaluate the performance of this chatbot and these pilot projects are included within this inventory.,Documentation is widely available,,,Yes,"Yes – agency has access to source code, but it is not public.", ,Yes,Enterprise Data Analytics and Visualization (EDAV) Platform,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, ChatCDC - CDC Enterprise Generative AI Chatbot (Ideation),Department of Health and Human Services,HHS,CDC,Mission-Enabling (internal agency support),,None of the above.,"CDC has an internal productivity tool available to all CDC staff which can be used in a variety of ways. This use case covers staff who will be using this general purpose to assist with the development of ideas or to overcome writers block in different tasks. When doing a task, staff can ask this tool to ask questions to help evolve any ideas being developed. Currently, CDC has internal Generative AI guidance determining how this tool can be used, and individual Centers, institutes, or Offices may implement their own policy to ensure safe and responsible AI. This tool enables CDC staff to have the productivity benefits of Generative AI powered chatbots without any of the security risks of 3rd party AI powered general purpose chatbots. This ensures staff can provide information regarding a current task along with general information with current tasks to help organize thinking, actions to take, and developing ideas for staff to use or reject as a part of their process. This reduces the time spent by staff dealing with situations such as writers block.","This chatbot, powered by Azure OpenAI Large Language Models, will generate responses to questions asked by the user. The chatbot's output can then be copied and pasted or manually entered into the appropriate testing process by the staff to ensure any code used from the chatbot's output meets the criteria of the current task. The output must be copied and pasted or manually entered somewhere for the output to be used by staff.",Operation and Maintenance,Neither,10/1/2023,11/1/2023,2/1/2024, ,Developed with both contracting and in-house resources.,N/A, , , ,No, ,No,No,No,,Yes. A series of pilot projects was conducted to evaluate the performance of this chatbot and these pilot projects are included within this inventory.,Documentation is widely available,,,Yes,"Yes – agency has access to source code, but it is not public.", ,Yes,Enterprise Data Analytics and Visualization (EDAV) Platform,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, ChatCDC - CDC Enterprise Generative AI Chatbot (Software Development),Department of Health and Human Services,HHS,CDC,Mission-Enabling (internal agency support),,None of the above.,"CDC has an internal productivity tool available to all CDC staff which can be used in a variety of ways. This use case covers staff who will be using this general purpose to assist with the development of software code which can be used for research, data analysis, software development, or other such systems. Currently, CDC has internal Generative AI guidance determining how this tool can be used, and individual Centers, institutes, or Offices may implement their own policy to ensure safe and responsible AI. When the AI chatbot is being used for coding assistance or software development, it affects several business processes related to agency operations and core task structures. Specifically, it can be utilized by various personnel within an organization such as the CDC, including software programmers using AI-specific code, those involved in encoding for software development, data analysts, scientific researchers conducting data analysis, data scientists implementing and testing new models, and software engineers developing or updating applications. The chatbot enhances productivity and efficiency across these roles by providing coding assistance and acting as a learning tool. For instance, data engineers may use the chatbot to generate PySpark code for their data transformation scripts within a larger pipeline. Additionally, individuals new to programming languages like Python can leverage the chatbot as a tutor to help them with tasks such as cleaning up datasets for evaluation purposes. In summary, the AI chatbot impacts business processes by aiding staff members in performing complex analytical tasks more efficiently and supporting their development work through guided coding assistance. This ultimately contributes to enhancing overall operations within the agency by streamlining research activities and internal process workflows.","This chatbot, powered by Azure OpenAI Large Language Models, will generate responses to questions asked by the user, in this case about either some software code, software code documentation, code concepts, or to help generate software code. The chatbot's output can then be copied and pasted or manually entered into the appropriate testing process by the staff to ensure any code used from the chatbot's output meets the criteria of the current task.",Operation and Maintenance,Neither,10/1/2023,11/1/2023,2/1/2024, ,Developed with both contracting and in-house resources.,N/A, , , ,No, ,No,No,No,,Yes. A series of pilot projects was conducted to evaluate the performance of this chatbot and these pilot projects are included within this inventory.,Documentation is widely available,,,Yes,"Yes – agency has access to source code, but it is not public.", ,Yes,Enterprise Data Analytics and Visualization (EDAV) Platform,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, ChatCDC - CDC Enterprise Generative AI Chatbot (Summarization),Department of Health and Human Services,HHS,CDC,Mission-Enabling (internal agency support),,None of the above.,"CDC has an internal productivity tool available to all CDC staff which can be used in a variety of ways. This use case covers staff who will be using this general purpose tool to summarize and synthesize information from larger documents such as meeting transcripts, research articles, or other documents. Currently, CDC has internal Generative AI guidance determining how this tool can be used, and individual Centers, institutes, or Offices may implement their own policy to ensure safe and responsible AI. This tool enables CDC staff to have the productivity benefits of Generative AI powered chatbots without any of the security risks of data uploads to third party tools. Using ChatCDC, Staff can have smaller, more easily digestible summaries to save staff time from reading larger documents if the time to read the full document is not available.","This chatbot, powered by Azure OpenAI Large Language Models, will generate responses to questions asked by the user, in this case a summary of a larger document. The chatbot's output can then be copied and pasted or manually entered into the appropriate testing process by the staff to ensure any code used from the chatbot's output meets the criteria of the current task.",Operation and Maintenance,Neither,10/1/2023,11/1/2023,2/1/2024, ,Developed with both contracting and in-house resources.,N/A, , , ,No, ,No,No,No,,Yes. A series of pilot projects was conducted to evaluate the performance of this chatbot and these pilot projects are included within this inventory.,Documentation is widely available,,,Yes,"Yes – agency has access to source code, but it is not public.", ,Yes,Enterprise Data Analytics and Visualization (EDAV) Platform,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, DCIPHER (SEDRIC) AIP for Advanced Foodborne Outbreak Investigation (AI Summarization),Department of Health and Human Services,HHS,CDC,Mission-Enabling (internal agency support),,None of the above.,"This AI solution impacts the process of investigating foodborne disease outbreaks. These foodborne outbreaks require cooperative efforts from CDC staff, FDA, USDA, and local agencies and the AI system is used through a centralized data platform System for Enteric Disease Response, Investigation, and Coordination (also known as SEDRIC). For more information on SEDRIC, please go to our website: https://www.cdc.gov/foodborne-outbreaks/php/foodsafety/tools/index.html SEDRIC's AIP use case provides CDC epidemiologists the ability to accelerate their investigations of multi-state foodborne disease outbreaks by more effectively leveraging data available in a rich data source, such as receipts from grocery stores, that otherwise requires extensive time and human effort to parse through. In addition, this workflow would free up epidemiologists' time and, potentially, increase the frequency with which both CDC and STLT partners could utilize shopper card, receipts, and free text responses to support investigations. Using the summarization capability, the extensive process of mapping common names of different food items is done automatically, greatly reducing the human labor time to generate dashboards of information regarding current foodborne investigations to serve as a decision point to aid in outbreak response.","The Artificial Intelligence Platform (AIP) available within SEDRIC provides CDC epidemiologists the power to accelerate their investigations of multi-state foodborne disease outbreaks. Using the information previously extracted from receipts and other records, AIP can summarize these results to provide insights from information pulled from shopper receipts. Given ingredients can be found in multiple food products, and some ingredients such as herbs like coriander/cilantro may go by multiple names or be reported in multiple languages, this summarization tool provides a faster way to gather summary information from receipts on different food items which may be part of a foodborne investigation.",Operation and Maintenance,Neither,10/1/2023,10/1/2023,2/1/2024, ,Developed with contracting resources.,N/A, , , ,No, ,No,No,No,,"Data are used in outbreak/response scenarios, such as foodborne illness outbreak response. Data used is dependent on the situation and outbreak, and may be owned by CDC, FDA, USDA, State Health Departments, Tribal Health Departments, Local Health Departments, Territorial Health Departments, or other entities.",Documentation has been partially completed,,,Yes,"Yes – agency has access to source code, but it is not public.", ,Yes,1 CDC Data Platform (1CDP),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, DCIPHER (SEDRIC) AIP for Advanced Foodborne Outbreak Investigation (Receipt Reading),Department of Health and Human Services,HHS,CDC,Mission-Enabling (internal agency support),,None of the above.,"This AI solution impacts the process of investigating foodborne disease outbreaks. These foodborne outbreaks require cooperative efforts from CDC staff, FDA, USDA, and local agencies and the AI system is used through a centralized data platform System for Enteric Disease Response, Investigation, and Coordination (also known as SEDRIC). For more information on SEDRIC, please go to our website: https://www.cdc.gov/foodborne-outbreaks/php/foodsafety/tools/index.html SEDRIC's AIP use case provides CDC epidemiologists the ability to accelerate their investigations of multi-state foodborne disease outbreaks by more effectively leveraging data available in a rich data source, such as receipts from grocery stores, that otherwise requires extensive time and human effort to parse through. In addition, this workflow would free up epidemiologists' time and, potentially, increase the frequency with which both CDC and STLT partners could utilize shopper card, receipts, and free text responses to support investigations. Manually entering receipt information, shopper card information, or other such free text field is traditionally error prone and time intensive for a variety of information. This AI system provides a human-in-the-loop opportunity to review and update data entry points while reducing time spent by staff to gain these insights. Having a set structured output as well increased standardization of this information and eases reporting in situations requiring cooperation from multiple organizations.","Palantir’s Artificial Intelligence Platform (AIP) available within SEDRIC provides CDC epidemiologists the power to accelerate their investigations of multi-state foodborne disease outbreaks. It can extract structured data from grocery receipts, shopper card records, and free-text responses in order to catalog the food items purchased by affected patients. It can also map those items to SEDRIC-defined vehicles which categorize the items and highlight commonalities across patients, helping to pinpoint potential outbreak vehicles.",Operation and Maintenance,Neither,10/1/2023,10/1/2023,2/1/2024, ,Developed with contracting resources.,N/A, , , ,No, ,No,No,No,,"Data are used in outbreak/response scenarios, such as foodborne illness outbreak response. Data used is dependent on the situation and outbreak, and may be owned by CDC, FDA, USDA, State Health Departments, Tribal Health Departments, Local Health Departments, Territorial Health Departments, or other entities.",Documentation has been partially completed,,,Yes,"Yes – agency has access to source code, but it is not public.", ,Yes,1 CDC Data Platform (1CDP),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, DFWED Food Vehicle Investigation Module,Department of Health and Human Services,HHS,CDC,Mission-Enabling (internal agency support),,None of the above.,"Foodborne outbreak investigation Outbreak investigation by providing estimates of output of potential food vehicles can increase speed of tracking foodborne outbreaks in terms of identification of hypothesis generation. This reduces operational burdens in foodborne response efforts.","NLP and Food Vehicle Investigation Module (FVIM) uses language learning models and Text Classification Tools to identify potential vehicles. NLP tool uses text classification to provide probabilities that a food is associated with the outbreak based on reports in the SEDRIC line list. FVIM identifies past outbreak with similar key demographic and temporal characteristics to the selected outbreak to guide hypothesis generation. FVIM analyses reported food exposures uploaded to the SEDRIC line list and categorizes responses into food commodity types that are then compared to FoodNet Population survey results weighted by age, sex and ethnicity and generates a statistical comparison",Operation and Maintenance,Neither,7/1/2022,7/1/2022,5/1/2023, ,Developed in-house., , , , ,No, ,Yes,No,No,,"Data are used in outbreak/response scenarios, such as foodborne illness outbreak response. Data used is dependent on the situation and outbreak, and may be owned by CDC, FDA, USDA, State Health Departments, Tribal Health Departments, Local Health Departments, Territorial Health Departments, or other entities.",Documentation is missing or not available,,,Yes,"Yes – agency has access to source code, but it is not public.", ,Yes, 1 CDC Data Platform (1CDP),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Genetic distance computation method for comparing complex multi-locus parasite (Cyclospora) genotypes,Department of Health and Human Services,HHS,CDC,Health & Medical,,None of the above.,"Investigating the similarity of infections used during epidemiologic investigations of cyclosporiasis outbreaks. It helps produce a clustering output for massive datasets comprising genotypes that are too complex to be analyzed using traditional genetic distance computation methods and aid in identifying similarities of infections when tracking cyclosporiasis outbreaks and other potential parasites.","The AI algorithm compares sets of haplotypes sequenced from a set of clinical samples and computes a set of genetic distances. It performs these comparisons across thousands of genotypes to identify infections that are closely related or unrelated. The resultant genetic distances are then clustered to define groups of closely related infections. This information is used routinely to complement epidemiologic investigations of cyclosporiasis (Cyclospora) outbreaks, and traceback investigations performed by state public health agencies and FDA. While this AI has mostly been applied to Cyclospora, we have also applied it to other parasites, including malaria, and certain parasitic worms.",Operation and Maintenance,Neither,1/1/2018,1/1/2018,9/1/2019, ,Developed in-house., , , , ,No, ,No,No,No,,"used Cyclospora sequence data that was generated by CDC, State Public Health Labs, and the Public Health Agency of Canada following a protocol for amplifying 8 genotyping markers. This protocol was developed by CDC. All sequence data generated by CDC and State Public Health Labs are publicly available through NCBI (https://www.ncbi.nlm.nih.gov/bioproject/PRJNA578931/). The sequence data from the Public Health Agency of Canada can typically be found on NCBI as well (https://www.ncbi.nlm.nih.gov/bioproject/PRJNA796535/).",Documentation is widely available,,,Yes,"Yes – agency has access to source code, but it is not public.", ,Yes,Advanced Molecular Detection Scientific Computing Platform,6-12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, MedCoder - Coding literal text cause of death information reported on death certificates to ICD-10,Department of Health and Human Services,HHS,CDC,Health & Medical,,None of the above.,"MedCoder is a subsystem of CDC's National Center for Health Statistics (NCHS) National Vital Statistics System for coding causes of death indicated on the death certificates to the International Classification of Diseases 10th Revision (ICD-10) codes. With the implementation of MedCoder, the percentage of deaths that can be automatically and accurately coded has risen from 70-75% using the previous coding system to over 85%. This improvement translates into substantial cost savings, amounting to hundreds of thousands of dollars that would have been spent on manual coding. Additionally, it significantly enhances the timeliness of data related to urgent public health concerns such as COVID and drug overdose deaths, enabling near real-time surveillance.","MedCoder assists with the identifying cause of death codes from death certificates using ICD-10 Underlying cause codes and automatically identifies and rejects complex causes of death as well as frequently miscoded ones, requiring manual coding or review. Additionally, in production, MedCoder utilizes natural language processing (NLP) to cleanse and standardize literal text cause of death statements before performing coding.",Operation and Maintenance,Neither,10/1/2017,2/1/2018,6/1/2022, ,Developed in-house., , , , ,No, ,No,No,Yes,,N/A,Documentation is widely available,Sex/Gender,,Yes,"Yes – agency has access to source code, but it is not public.", ,Yes,National Vital Statistics System (NVSS),More than 12 months,Yes,,No,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, NCIRD SmartFind ChatBots - Public and Internal ,Department of Health and Human Services,HHS,CDC,Mission-Enabling (internal agency support),,None of the above.,"Internal partner mailbox email management including managing knowledge base. Previously, this included public facing chatbots but the public ChatBots are no longer active. This uses previously cleared FAQs and other reviewed information accessible publicly as the knowledge base. The internal Knowledge-Bot's SharePoint component is helping program staff manage emails from partners more efficiently and effectively including creation and maintenance of a knowledge base that can be utilized across staff managing the mailbox. ","Develop conversational ChatBots (Public Flu, Public COVID-19 Vaccination, Internal Knowledge-Bot) that analyze free text questions entered by the public, healthcare providers, partners, and internal staff, and provide agency-cleared answers which best match the question. Developed in collaboration with Microsoft staff during COVID-19 pandemic using their Cognitive Services, Search, QnA Maker, Azure Healthcare Bot, Power Automate, SharePoint, and webapps. ",Operation and Maintenance,Neither,10/1/2020,3/1/2024,12/1/2024, ,Developed with both contracting and in-house resources.,N/A, , , ,No, ,No,No,No,,"For the public-facing chatbots, we use public-facing FAQs.",Documentation is complete,,,Yes,"Yes – agency has access to source code, but it is not public.", ,Yes,CDC OD ATO,Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, NIOSH Industry and Occupation Computerized Coding System (NIOCCS),Department of Health and Human Services,HHS,CDC,Mission-Enabling (internal agency support),,None of the above.,"Replaces manual coding of industry and occupation text to standardized codes so that they can be used for research and analysis. Reduces the high cost of manually coding industry and occupation data while promoting increased coding speed, accuracy, and consistency.",Uses machine learning models to translate industry and occupation text into standardized codes that can be used for research and analysis.,Operation and Maintenance,Neither,10/1/2024,10/1/2024,1/1/2024, ,Developed in-house., , , , ,Yes,This use case is operated in compliance with HHS's agencywide Information Quality Act policy and procedures.,No,No,Yes,,STLT's death record data (received via NCHS) and BRFSS survey data,Documentation has been partially completed,,,Yes,"Yes – agency has access to source code, but it is not public.", ,Yes,Modernization Platform NIOSH (MPN),Less than 6 months,Yes,,No,,Yes,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Nowcasting Injury Trends ,Department of Health and Human Services,HHS,CDC,Health & Medical,,None of the above.,"Predicting current injury rates and real time estimates of injuries and deaths. The internal dashboard that will be created as a result of this effort will enhance the understanding of injury trends and expedite surveillance and research activities. This initiative will have a direct impact on the speed at which emerging trends in injuries resulting in deaths are identified and investigated. Gold standard data take time to process and be made available. Real-time insight of injury trends when gold standard data are not available can be a resource resulting in timelier situational awareness to inform injury surveillance efforts. ","An internal-facing, interactive dashboard incorporating multiple traditional and non-traditional datasets and a multi-stage machine learning pipeline to 'nowcast' injury death trends nationally on a week-to-week basis. ",Operation and Maintenance,Neither,1/1/2021,1/1/2021,1/1/2022, ,Developed in-house., , , , ,No, ,No,No,No,,Emergendy Department data from the National Syndromic Surveillance Program,Documentation is widely available,,,Yes,"Yes – agency has access to source code, but it is not public.", ,Yes,Enterprise Data Analytics and Visualization (EDAV) Platform,6-12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Risk Assessment Module (RAM) for the National Diabetes Prevention Program (National DPP) Operations Center.,Department of Health and Human Services,HHS,CDC,Mission-Enabling (internal agency support),,None of the above.,"Tracking program progress. This AI assists in determining if an organization that is currently operating within the National DPP is at risk of improperly starting, at risk of going inactive, or at risk of not achieving the goals necessary to continue successfully in CDC’s Diabetes Prevention Recognition Program. The RAM module will help program managers synthesize large amounts of organization-level data in making decisions on how best to assist organizations with their programs, ultimately leading to increased program participation and improved health outcomes in program participants. ","The RAM is a reporting tool within the National DPP Operations Center which already has an ATO. The RAM module ingests large amounts of organization-level data including National DPP participant enrollment records, demographics, and specific risk factors to perform supervised learning and model generation. This machine learning model is used to predict and quantitatively rank those organizations that are at highest risk of failing to meet program objectives. This information, both input and RAM output, is currently fully sequestered to associates of the National DPP of the CDC. In the future, with continued development and validation, the plan is to allow external-to-CDC State Quality Specialist (SQS) users access to the outputs of RAM for additional assistance with improving organization success.",Operation and Maintenance,Neither,8/1/2023,9/1/2023,8/1/2024, ,Developed with contracting resources.,N/A, , , ,No, ,No,No,No,,DDT DPRP Portal - historical data from organization's 6-mth submissions of participant attendance of Lifestyle change classes.,Documentation is missing or not available,,,Yes,"Yes – agency has access to source code, but it is not public.", ,Yes,DDT DPRP Portal,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Semi-Automated Nonresponse Detection for Surveys (SANDS),Department of Health and Human Services,HHS,CDC,Mission-Enabling (internal agency support),,None of the above.,"SANDS is a fine-tuned large language model (LLM) designed to assist with human-in-the-loop procedures specifically for open-ended survey responses. Open-ended survey responses vary in quality and are challenging to use effectively because they are labor-intensive to review and therefore cost-prohibitive to use at scale. SANDS is intended to reduce the burden by filtering out clear and obvious non-response such as ""This is so dumb, why I am I wasting my time on such a stupid question"" or ""Because It is true"" which provide no value to researchers. This LLM is available on HuggingFace and is not routinely updated, but it can be used like any open-source LLM through secure instances of Python. Manual curation of open-ended survey responses is time-consuming, often requiring extensive hours to identify themes and review outputs. SANDS significantly reduces this manual burden by providing scores for responses, enabling researchers to quickly compile an initial high-quality dataset for qualitative research. SANDS also flags responses needing further examination, streamlining the review process.",CDC's National Center for Health Statistics (NCHS) has developed and released a model to detect nonresponses in open-text survey responses. This helps improve survey data quality and question and questionnaire design. The system is a natural language processing (NLP) model that has been fine-tuned on a custom dataset of survey responses. ,Operation and Maintenance,Neither,6/1/2021,9/1/2021,9/1/2022, ,Developed in-house., , , , ,No, ,Yes,No,No,,"3,000 labeled open-ended responses to web probes on questions relating to the COVID-19 pandemic gathered from the Research and Development Survey or RANDS conducted by the Division of Research and Methodology at the National Center for Health Statistics",Documentation is widely available,,,Yes,"Yes – agency has access to source code, but it is not public.", ,No, ,Less than 6 months,Yes,,Yes,,No,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, Sequential Coverage Algorithm (SCA) and partial Expectation-Maximization (EM) estimation in Record Linkage,Department of Health and Human Services,HHS,CDC,Health & Medical,,None of the above.,"CDC's National Center for Health Statistics (NCHS) Data Linkage Program has developed a record linkage program designed to maximize the scientific value of the Center’s population-based surveys. Linked data files enable researchers to examine the factors that influence disability, chronic disease, health care utilization, morbidity, and mortality. The linkage algorithms initially used by the program did not implement machine learning in either deterministic or probabilistic linkages and required several manual steps for these processes.  Using machine learning in the CDC's National Center for Health Statistics (NCHS) Data Linkage Program offers several benefits. Machine learning algorithms help improve the accuracy and efficiency of data linkage by developing joining methods or blocking groups. Working with very large datasets can be time-consuming and resource-intensive. By implementing supervised and unsupervised machine learning techniques, the Data Linkage Program can automate the process of developing and selecting blocking groups, reducing manual effort and increasing efficiency. This additionally improves the scalability of data linkages. Machine learning algorithms can continuously learn and adapt based on new data inputs and feedback. This allows the Data Linkage Program to refine their linkage algorithms over time.","CDC's National Center for Health Statistics (NCHS) Data Linkage Program has implemented both supervised and unsupervised machine learning (ML) techniques in their linkage algorithms. The Sequential Coverage Algorithm (SCA), a supervised ML algorithm, is used to develop joining methods (or blocking groups) when working with very large datasets. The unsupervised partial Expectation-Maximization (EM) estimation is used to estimate the proportion of pairs that are matches within each block. Both methods improve linkage accuracy and efficiency.",Operation and Maintenance,Neither,2/1/2019,8/1/2019,8/1/2020, ,Developed with contracting resources.,N/A, , , ,No, ,Yes,No,No,,"Data used to evaluate the models include data from the National Hospital Care Survey, the National Health and Nutrition Examination Survey, and the National Health Interview Survey as well as linked administrative data.",Documentation is widely available,Sex/Gender,,Yes,"Yes – agency has access to source code, but it is not public.", ,Yes,Consolidated Statistical Platform,6-12 months,Yes,,Yes,,Yes,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, TowerScout: Automated cooling tower detection from aerial imagery for Legionnaires' Disease outbreak investigation,Department of Health and Human Services,HHS,CDC,Health & Medical,,None of the above.,"During Legionella outbreaks, the National Center for Immunization and Respiratory Diseases (NCIRD)/Division of Bacterial Diseases (DBD) needs to identify cooling towers that could potentially be spreading Legionella bacteria. Using AI object detection, TowerScout detects cooling towers approximately 600 times faster than manual searches, allowing researchers to more efficiently respond during outbreaks. ",TowerScout uses object detection and image classification models to detect cooling towers within aerial imagery. ,Operation and Maintenance,Neither,1/1/2021,1/1/2021,5/1/2021, ,Developed with contracting resources.,N/A, , , ,No, ,No,No,No,,None,Documentation is widely available,,,Yes,Yes – source code is publicly available.,https://github.com/TowerScout/TowerScout,Yes,Enterprise Data Analytics and Visualization (EDAV) Platform,Less than 6 months,Yes,,Yes,,No,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, Creation of synthetic Survey-like Insurance Names for use coding NHIS private insurance responses,Department of Health and Human Services,HHS,CDC,Health & Medical,,None of the above.,"This solution will impact the process of manual coding and reviewing text errors in the National Health Interview Survey Previous efforts to augment the manual coding have proven ineffective. Given the complex nature of insurance programs, this results in over 200 FTE hours a year being spent on just the initial coding of open text insurance fields. Additional hours are spent reviewing the manual coding for errors. By creating survey-like insurance responses, this could theoretically save over 100 hours of staff labor a year, greatly increasing efficiency and timeliness.","The National Health Interview Survey calculates insurance coverage rates among the US non-institutionalized resident population, including statistics of the percentage of people with private insurance coverage. The collection of information on insurance coverage is accomplished in two parts. First, survey respondents self-identify the type of insurance they have. Secondly, to aid in the verification of insurance type, survey respondents also provide the name of the health plan or program they have in an open text field. The information collected in the open text field may include misspellings, acronyms, and rarely match exactly the true complete insurance plan name. This leads to a time intensive process as staff may need to decipher the plan name in the open text field mentioned by the respondent and/or abbreviated by the interviewer and if that plan name confirms the correct type of insurance initially indicated during the interview process. By creating AI generated known acronyms as data points for projects, as an example Blue Cross Blue Shield of Alabama to BCBS of AL and BC/BS of AL, this will create a more representative set of information for staff to use when coding potential saving up to hundreds of FTE hours reviewing responses. This also works in combination with existing efforts to augment manual coding with AI and Machine Learning to increase efficiency. ",Retired,Neither, , , ,6/1/2024, , , , , , , , , , ,, , ,,, , , , , ,, ,, ,,,, , ,,,,,,,,,,,,,,,,,, Development of in-silico genomic and patient datasets using generative ML algorithms,Department of Health and Human Services,HHS,CDC,Mission-Enabling (internal agency support),,None of the above.,"This solution will impact the process of finding reference genomes Reduce the development time of custom tools, finding reference genomes, and additional administrative burdens. The datasets developed would be beneficial in testing out bioinformatic pipelines. ","This solution uses NLP generative AI toolsets (e.g. ChatGPT, co-pilot, etc.) to generate code for the development of bacterial and viral in-silico genomic sequences and patient data. This will be compared to already established pipelines to determine factors such as code viability, data accuracy, code licensing, and any drawbacks and/or advantages these services provide. For example, we would ask GPT-4 a prompt, ""generate python code to create Sars COV2 in-silico fastq sequencing data."". We will look at where did it reference the code base to generate the code and is it open source or have licensing requirements. ",Retired,Neither, , , ,2/1/2024, , , , , , , , , , ,, , ,,, , , , , ,, ,, ,,,, , ,,,,,,,,,,,,,,,,,, Mastering Metadata: AI-Powered Governance for Data Insights,Department of Health and Human Services,HHS,CDC,Mission-Enabling (internal agency support),,None of the above.,"The implementation of this solution will have a significant influence on the maintenance of data quality and data assets on the data.cdc.gov platform. Accurate, reliable, and timely data available in data.cdc.gov is critical in making meaningful public health decisions. This publicly available data repository is used worldwide by researchers, policy makers, and general public to access, manage, analyze, and visualize public health data. Through this project, we will be able to scale up the magnitude and quality of datasets from NCIRD that’s made available via data.cdc.gov and thus maximizing public health impact for a diverse audience at multiple levels.","data.cdc.gov serves as a publicly accessible repository for CDC data, utilized by researchers, policymakers, and the general public to monitor and comprehend public health trends. Ensuring the accuracy and reliability of the data on data.cdc.gov is crucial for informed decision-making in public health. To achieve this, the proposed solution involves leveraging AI to maintain CDC's National Center for Immunization and Respiratory Diseases data assets on data.cdc.gov to automate various tasks such as checking metadata completeness, validating metadata against a schema, comparing metadata with other sources, and monitoring metadata changes over time.",Retired,Neither, , , ,6/1/2024, , , , , , , , , , ,, , ,,, , , , , ,, ,, ,,,, , ,,,,,,,,,,,,,,,,,, PII detection using Private AI ,Department of Health and Human Services,HHS,CDC,Mission-Enabling (internal agency support),,None of the above.,"Numerous data sources contain personally identifiable information (PII) and protected health information (PHI), which cannot be publicly published or shared among federal partners due to confidentiality concerns. This includes sensitive records like death certificates, survey responses, and electronic health records. To enable broader utilization of this data and enhance insights, a PII detection system was explored in this project. Specifically, the availability of an off-the-shelf tool called Private AI was investigated for identifying PII. However, it is important to note that the license for this tool expired after one year and there are no current plans to renew the contract. At present, the presence of PII within data and the labor intensive process to ensure PII redaction and removal has prevented broader sharing of data available within CDC. However, implementing an automated system which performs as well as humans and more quickly could significantly reduce the time required for human review, enabling faster dissemination of crucial public health information and data sources.","CDC's National Center for Health Statistics (NCHS) has been assessing the NLP solution provided by Private AI, which is specifically designed to detect, mask, and substitute personally identifiable information (PII) within textual data. This collection of models aims to securely identify and eliminate PII from unstructured text datasets across various platforms within the CDC network.",Retired,Neither, , , ,3/1/2024, , , , , , , , , , ,, , ,,, , , , , ,, ,, ,,,, , ,,,,,,,,,,,,,,,,,, Reddit Post Analysis for Sexual Health Using Large Language Models,Department of Health and Human Services,HHS,CDC,Health & Medical,,None of the above.,"The AI will impact the data extraction and analysis processes for understanding how users seek sexual health advice online using Reddit. In additional to traditional public health impacts this can include reducing administrative burdens and other ancillary benefits. This is an important criteria when selecting the initial round of projects. Understand how and why people seek sexual health advice online. This may lead to finding gaps in online health resources, identifying underserved populations who resort to using online resources, and discovering changes in sexual health behavior over time.","Reddit is a social media website where users can submit and respond to posts around a wide variety of topics. Reddit is organized into subcommunities called subreddits, and many people turn to sexual health focused subreddits, such as the “r/STD” subreddit, for advice. The Reddit posts can be a valuable source of text and image data since these posts reveal immediate sexual health concerns and questions that may be overlooked. However, the volume of posts makes large scale manual analysis impractical, and extracting insights using conventional methods is difficult. This solution uses large language models (LLMs) to assist in and automate data extraction and analysis to better understand how users seek sexual health advice online.",Retired,Neither, , , ,2/1/2024, , , , , , , , , , ,, , ,,, , , , , ,, ,, ,,,, , ,,,,,,,,,,,,,,,,,, Retrieval Augmented Generation (RAG) with Q-Bank,Department of Health and Human Services,HHS,CDC,Mission-Enabling (internal agency support),,None of the above.,"This solution will impact the process of searching CCQDER’s Q-Bank research publications on question evaluation for surveys. Q-Bank reports provide context needed to better understand and interpret data from survey estimates, as well as information about question performance and fit-for-use in various data collection contexts. This project aims to improve access and visibility of CCQDER’s Q-Bank research publications on question evaluation for surveys. Additionally, indexing Q-Bank as part of this project should help to relieve some of the administrative burden of tagging and organizing reports.","This solution uses Retrieval Augmented Generation to develop an LLM-based search tool based on the Collaborating Center for Questionnaire Design and Evaluation Research’s (CCQDER) Q-Bank reports. The main motivation of the solution is to make it easier to navigate and identify reports relevant to a user’s research interests. Because the reports are already publicly available and they represent a relatively small corpus, they provide an excellent opportunity to develop prototype AI tools.",Retired,Neither, , , ,6/1/2024, , , , , , , , , , ,, , ,,, , , , , ,, ,, ,,,, , ,,,,,,,,,,,,,,,,,, Using generative AI to gain insight of older adult falls,Department of Health and Human Services,HHS,CDC,Health & Medical,,None of the above.,"This solution will impact the process of extracting information from medical narratives Traditional methods of data and information extraction from medical narratives involve manual scrutiny and the use of rule-based models. These methods are time-consuming and can overlook crucial details. By utilizing generative AI, a more comprehensive understanding of the circumstances surrounding injuries from falls can be achieved. Understanding the specific causes, circumstances, and outcomes of falls can lead to more targeted public health interventions.",This solution extracts insights regarding older adult falls injuries from Emergency Department medical narratives and testing out methods submitted. The data source to be used is the National Electronic Injury Surveillance System (NEISS) data.,Retired,Neither, , , ,6/1/2024, , , , , , , , , , ,, , ,,, , , , , ,, ,, ,,,, , ,,,,,,,,,,,,,,,,,, AI-assisted comment triaging tool,Department of Health and Human Services,HHS,CMS,Mission-Enabling (internal agency support),,None of the above.,"Annual rulemaking-- the Physician Fee Schedule receives thousands of public comments and require assistance with reviewing and assigning the comments to the particular topics they address. Assists with workload for the PFS team","We use AI to assist in comment review, such as identifying form letters. ",Operation and Maintenance,Neither,6/1/2022,12/1/2021,6/1/2022,,Developed with contracting resources.,N/A,No,,,No,,No,No,Yes,,None,Documentation is complete,,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,No,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Bankruptcy BOT,Department of Health and Human Services,HHS,CMS,Mission-Enabling (internal agency support),,None of the above.,"The Bankruptcy/Overpayment Division of IFM is responsible for carrying out the Agency's oversight responsibilities for our contractors' debt management functions, and for protecting Medicare's interest in debt recoupment activities complicated by bankruptcy. A timely identification of filed provider bankruptcies allows Medicare to secure recovery rights before timeframes to file a proof of claim expire.",The Bankruptcy BOT proactively identifies bankruptcy case with Medicare interest.,Operation and Maintenance,Neither,1/1/2020,1/1/2020,9/1/2021,,Developed with contracting resources.,47QRAA18D001P,No,,,No,,Yes,No,Yes,,PECOS and BCRS,Documentation is missing or not available,Residency Status,,No,"Yes – agency has access to source code, but it is not public.",,Yes,PACER,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Central Data Abstraction Tool-Modernized (Modernized-CDAT)- Intake Process Automation (PA) Tool,Department of Health and Human Services,HHS,CMS,Law & Justice,,None of the above.,"Intake PA uses advanced capabilities (NLP, OCR, AI, ML) to automate, modernize, and reduce manual efforts related to medical record review functions within MA RADV audits Eliminates manual review of Medical Record (MR) submissions, expedites the Intake process and realizes cost savings in both time and resources.",The PA-BOT tool will replace the RADV audit Intake manual functions for Steps 2 & 3 of the RADV audit process. The RADV Intake process conducts a review of the MAO Medical Record (MR) submissions and validates that the MR submissions from MAOs have met RADV audit intake requirements and can proceed to MR abstraction/coding.,Operation and Maintenance,Neither,9/1/2017,11/1/2017,4/1/2019,,Developed with contracting resources.,N/A,No,,,No,,Yes,No,No,,"Yes, prior RADV results were used to train the models.",Documentation is widely available,,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Central Data Abstraction Tool - Modernized (CDAT-M),Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, CMS Enterprise Portal Services (CMS Enterprise Portal-Chatbot),Department of Health and Human Services,HHS,CMS,Mission-Enabling (internal agency support),,None of the above.,"CMS Enterprise Portal AI for Process Efficiency Improvement| Knowledge Management - Empowers users to find information or troubleshoot issues more efficiently - Reduces load on system help desks - Provides better insight into customer inquiries and issues informing process and design improvements "," - Automated text-based bot providing support to Portal users - Uses machine learning via Amazon Lex to answer a user’s question - Takes user input and finds the best match to the question or problem statement - Automates tasks such as: customer support; processing of user questions; and solving user problems/issues",Operation and Maintenance,Neither,8/1/2020,12/1/2020,4/1/2021,,Developed with contracting resources.,N/A,No,,,No,,No,No,No,,CMS Enterprise Portal and CMS Identity Management (IDM) user guides,Documentation is widely available,,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,CMS Enterprise Portal,Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, Enhanced Direct Enrollment Outlier Detection,Department of Health and Human Services,HHS,CMS,Mission-Enabling (internal agency support),,None of the above.,"Enhanced Direct Enrollment (EDE) allows consumers to apply for and enroll in an exchange plan directly through an approved partner’s UI, without being redirected through the Healthcare.gov application. These partner systems directly interface with the APIs developed by the FFE. As EDE Partners gain more control over their application process, the FFE must ensure program integrity. Ensure FFE EDE program integrity","This use-case will implement Machine Learning to identify anomalies/quality issues with partner-submitted person, application, and policy data​.",Operation and Maintenance,Neither,7/1/2019,1/1/2020,1/1/2020,,Developed with contracting resources.,N/A,No,,,No,,No,No,Yes,,CMS,Documentation is widely available,,,Yes,No – agency does not have access to source code.,,Yes,FFM,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Exchange Complaints Review Categorization ML/AI ,Department of Health and Human Services,HHS,CMS,Law & Justice,,None of the above.,"The Exchange Complaints Review Contractor (ECRC) receives hundreds of consumer complaints from the Federally-facilitated Exchange (FFE) call center daily, who then must manually read each complaint and categorize each complaint into one or more categories according to the Complaint Review Standard Operating Procedure (SOP). The categorization assists CPI in determining the nature of the complaint, the action necessary, and the component within CMS who the complaint should be routed to. In general, each complaint can take up to 10 minutes to review and categorize, including a quality review component. The output (list of all complaints categorized) allows CMS to expediate routing the complaint to the appropriate CMS caseworker or Issuer to take the appropriate action necessary to make the consumer whole and ensure they can receive the necessary medical treatment. The output when done by AI is done in a matter of minutes, versus weeks when completed by human reviewers. ","CPI utilizes machine learning/artificial intelligence (AI) and continuously feeds the complaints review model thousands of complaints in order to teach the model how to accurately categorize the complaints, based on keywords. This is done in a matter of minutes instead of weeks by a human complaint reviewer.",Operation and Maintenance,Neither,1/1/2022,1/1/2022,10/1/2024,,Developed with both contracting and in-house resources.,N/A,No,,,No,,Yes,No,Yes,,"HICS, Marketplace",Documentation has been partially completed,,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,MIDAS,6-12 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Feedback Analysis Solution (FAS),Department of Health and Human Services,HHS,CMS,Mission-Enabling (internal agency support),,None of the above.,"The Feedback Analysis Solution is a system that uses CMS or other publicly available data (such as Regulations.Gov) to review public comments and/or analyze other information from internal and external stakeholders. The FAS uses Natural Language Processing (NLP) tools to aggregate, sort and identify duplicates to create efficiencies in the comment review process. FAS also uses machine learning (ML) tools to identify topics, themes and sentiment outputs for the targeted dataset. FAS help categorize stakeholder comments or feedback (collected in multiple venues), thereby enabling analysts to use the system to quickly identify comments that may impact program/policy decisions.","Feedback Analysis Solution (FAS) uses AI/ML/NLP tools to identify topics, themes, and sentiments outputs for the target datasets. Groups and offices within CCSQ use the solution in slightly different ways to ingest unstructured data, interpret and structure it for use in ML models after de-duplication of comments.",Operation and Maintenance,Neither,1/1/2021,9/1/2021,9/1/2021,,Developed with both contracting and in-house resources.,75FCMC22A0018/75FCMC23F0001,No,,,No,,No,No,Yes,,CMS,Documentation is widely available,,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,CDRAP,6-12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, IT System Utilization Optimization,Department of Health and Human Services,HHS,CMS,Mission-Enabling (internal agency support),,None of the above.,"The FFE application usage patterns vary and are dependent on differing environment usage periods. CMS resources are currently manually scaled, not allowing immediate actions in correlation with usage changes. Automation of application scaling",This use-case will implement Machine Learning to determine optimized infrastructure/application scaling to support system volume.,Operation and Maintenance,Neither,7/1/2019,1/1/2020,1/1/2020,,Developed with contracting resources.,N/A,No,,,No,,No,No,Yes,,CMS,Documentation is widely available,,,Yes,No – agency does not have access to source code.,,Yes,FFM,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Knowledge Management Platform,Department of Health and Human Services,HHS,CMS,Mission-Enabling (internal agency support),,None of the above.,"The business process it affects involves accessing and using structured and unstructured data sources to enable data-driven decision-making.  It helps by creating efficiencies, reducing burden, and generating an increased return on investment in the achievement of CMS's mission and business objectives.","It leverages natural language processing, knowledge graphs, and large language models to make the knowledge within structured and unstructured data sources accessible and meaningful to users.",Operation and Maintenance,Neither,9/1/2021,10/1/2021,11/1/2021,,Developed with contracting resources.,KMP / SEAS IT,No,,,No,,No,No,No,,CMS,Documentation has been partially completed,,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,CEDAR,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Risk Adjustment Outlier Analysis,Department of Health and Human Services,HHS,CMS,Mission-Enabling (internal agency support),,None of the above.,"The RA program spreads the financial risk borne by Issuers due to offering a variety of plans meeting the need of the diverse population. RA payments are distributed based on population risk levels. The FFE uses a distributed data solution to calculate plan average actuarial risk and associated payments and must avoid potential impacts to annual calculations. Maintain RA Program risk integrity",This use-case will implement Machine Learning to identify outliers in issuer data / behavior that may unduly influence risk adjustment payments.,Operation and Maintenance,Neither,7/1/2024,6/1/2024,1/1/2020,,Developed with contracting resources.,N/A,No,,,No,,No,No,No,,CMS,Documentation is widely available,,,Yes,No – agency does not have access to source code.,,Yes,FFM,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Agent/Broker Fraud Analysis,Department of Health and Human Services,HHS,CMS,Mission-Enabling (internal agency support),,None of the above.,"Agent/Brokers (A/Bs) support the consumer enrollment and eligibility process. Because of this, they have learned the intricate details of the FFE for accessing applications, submitting eligibility determinations, and adding enrollments to their line of business, opening up the possibility of fraud. Improves the oversight and integrity of Marketplace Applications and Enrollments assisted by Agents and Brokers. Helps CCIIO identify suspicious behaviors to refer to CPI.",Utilizes Machine Learning to identify suspicious behaviors of Agents and Brokers in the Marketplace,Operation and Maintenance,Neither,7/1/2024,1/1/2024,1/1/2024,,Developed with contracting resources.,N/A,No,,,No,,Yes,No,No,,CMS,Documentation is widely available,,,Yes,No – agency does not have access to source code.,,Yes,FFM,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, CCSQ ServiceNow AI Search,Department of Health and Human Services,HHS,CMS,Government Services (includes Benefits and Service Delivery),,None of the above.,"CCSQ ServiceNow AI Search is a product inside ServiceNow (SaaS). It replaces traditional Zing search tool (exact search), and enables users with more flexible searches to get relevant and actionable answers quickly. * Improve search relevance * Promote self-service by empowering users to find information independently, and potential reduce number of cases","AI Search will * Display most relevant results first * Support synonyms, auto-corrections, stop words, and auto-completion AI Search analytics will provide insights into search usage, performance, trends, metrics, and how to improve search experiences. Plan to do AI Search tune-up next",Operation and Maintenance,Neither,11/1/2023,2/1/2024,4/1/2024,,Developed with contracting resources.,N/A,No,,,Yes,This use case is operated in compliance with HHS's agencywide Information Quality Act policy and procedures.,No,No,No,,"ServiceNow knowledge articles, ideas, catalog items data",Documentation is widely available,,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,QSC,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, MSP Assignment Bot,Department of Health and Human Services,HHS,CMS,Mission-Enabling (internal agency support),,None of the above.,"This impacts automatically retrieving records from a contractor's system, and importing them into an internal OPOLE Access database. Once populated in the database the record is assigned to staff to review and adjudicate. Pulls more cases out of BCRC and assigns much faster than an analyst. By allowing BOT to pull and assign cases instead of an analyst, this approach provides maximum time for staff to process cases which decreases the response time.",The Medicare Secondary Payer Operations Group (MSPOG) uses a BOT to pull compromise cases out of BCRC and assign to staff in our internal Management Information System (MIS),Implementation and Assessment,Neither,3/1/2022,4/1/2022,8/1/2022,,Developed in-house.,,No,,,No,,Yes,No,Yes,,case turn around times,Documentation is complete,,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, "Relationships, Events, Contacts, and Outreach Network (RECON)",Department of Health and Human Services,HHS,CMS,Mission-Enabling (internal agency support),,None of the above.,"RECON AI for Recommender System| Sentiment Analysis It has the potential to analyze data faster.","This was “native” Salesforce AI capabilities, not anything custom – and that platform-level AI is not being used in RECON at this time.",Retired,Neither,,,,2/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Complaint Analysis POC,Department of Health and Human Services,HHS,CMS,Health & Medical,,None of the above.,"The business process this AI could affect involves the analysis of complaint data used by the OPOLE team. It could help the OPOLE team to understand the main issues and potential root causes of the complaints, enabling them to address these issues more efficiently and effectively.",It utilized data science techniques and an LLM to analyze complaint data.,Initiated,Neither,2/1/2024,,,,,,No,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Data Lake/Load-Extract-Transform-Load (L-ETL),Department of Health and Human Services,HHS,CMS,Mission-Enabling (internal agency support),,None of the above.,"CMS is using Security Data Lake to modernize the Load-Extract-Transform-Load (L-ETL) pipelines and data tooling. There is no actual ML/AI work being done here today, rather, we are beginning work on the scaffolding that will open up these opportunities in 1-2 years time.  Building on top of a modern data platform will provide opportunities to experiment with machine learning model development against this data--solving any number of problems that require decisions to be made about inferences over time series data. ","CMS will be enhancing Agency security to bring together more system, telemetry and program data in one place with a unifying governance model.",Initiated,Neither,6/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, FOIA Document Review/Redaction Automation,Department of Health and Human Services,HHS,CMS,Mission-Enabling (internal agency support),,None of the above.,"OIT & OSORA partnership: Internal business process of reviewing and redacting CMS business documents subject to FOIA requests. Reduces burden on FOIA staff to manually redacting business documents subject to FOIA requests.",Utilizes GenAI technology to automate review and redaction of CMS documents per FOIA requests/process.,Initiated,Neither,7/1/2024,,,,,,No,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Help Desk Responses,Department of Health and Human Services,HHS,CMS,Government Services (includes Benefits and Service Delivery),,None of the above.,"The Division of Issuer Management and Operations has talked with their contractor, LMI, about using an AI tool for the Help Desk contract. This would reduce the amount of time staff contractors need to generate answers for SME review and approve responses to QHP issuers and other external entities that ask questions to the help desk. ",This AI feature in the issuer help desk would help generate responses to common questions from issuers and external organizations based on previously cleared material.,Initiated,Neither,12/1/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, 508 Accessibility Automation for Documents,Department of Health and Human Services,HHS,CMS,Mission-Enabling (internal agency support),,None of the above.,"OIT: Internal business process at individual employee level to automate review, identification, and correction of potential 508 accessibility issues in documents. Reduces burden associated with making documents 508 compliant (for individual CMS employees).",Utilizes GenAI technology to automate review and generation of corrected documents compliant with 508 accessibility rules.,Initiated,Neither,10/1/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Brand vs Generic Market Share,Department of Health and Human Services,HHS,CMS,Health & Medical,,None of the above.,"The Drug Data Processing System (DDPS) needs to predict changes in Medicare Part D costs Understanding beneficiary changes from Brand to Generic drugs allows for more accurate financial estimates.",Analyze generic drugs compared to brand drugs over time and forecast future market shares based on Part D claims volume,Initiated,Neither,10/1/2021,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Chat Client - Resource Library,Department of Health and Human Services,HHS,CMS,Education & Workforce,,None of the above.,"Continue to assess and refine content present in both the informational public side of QPP, Resource Library, Webinar Library, and Top Navigation as the site continues to grow. The Chat Client would benefit the program by improving user experience through easier access to information, increased program understanding, and ability to communicate in multiple languages. ","Powered by Amazon Bedrock, by loading all resources, files, references etc. available to the QPP user community, we can establish a knowledge base that allows the bot to scan all available program documentation to construct a message that accurately answers a user's question. ",Initiated,Neither,8/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Chatbot within Hub,Department of Health and Human Services,HHS,CMS,Government Services (includes Benefits and Service Delivery),,None of the above.,"Currently the following help support activities are handle by a person: Personalized FAQ’s; Clarify questions with schema and onboarding etc.; Handle - Where is “My file” or “My Request” inquiries Provide a - Talk to Agent feature Scheduling a testing window Provide data summarization and reporting for internal stakeholders Report operational health of the system Include training materials, Q/A about the system and such. Help support team in day-to-day communication with external partners","Build a chatbot that will address some help support activities through: Access to Internal Knowledge base Including RAG (Retrieval Augmented Generation). Personalized FAQ’s & Contextual generation. Interaction with Live Agent. Chat & Talk Feature. Ability to query custom data source for File or Case status.",Initiated,Neither,9/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Drug cost anomaly detection,Department of Health and Human Services,HHS,CMS,Health & Medical,,None of the above.,"CMS Division of Payment Reconciliation is responsible for monitoring Prescription Drug Event (PDE) data for accuracy Allows for identification of outliers in PDE which can be corrected through Part D plan outreach in order to prevent overpayments.",Identify anomalies in drug costs on Part D claims,Initiated,Neither,10/1/2021,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Drug Cost Increase Predictions,Department of Health and Human Services,HHS,CMS,Health & Medical,,None of the above.,"The Drug Data Processing System (DDPS) needs to predict changes in Medicare Part D costs Machine Learning model uses multiple methods to predict changes to Part D drug costs based on market changes.",Use Historical drug costs increases to predict future increases,Initiated,Neither,10/1/2021,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Medicare Part C/D Marketing Material Review,Department of Health and Human Services,HHS,CMS,Mission-Enabling (internal agency support),,None of the above.,"Drug and Health Plan Operations (DHPO) staff reviews both advertising mediums and channels materials that includes over 22 different types of media marketing materials ranging from written, digital, and more. The marketing materials are submitted to CMS from either the Medicare Health Plans and/or their third party marketing organizations. DHPO staff reviews these marketing materials to ensure they meet Medicare Marketing Guidelines. The current process is time consuming and resource intensive. This reduces the efficiency and quality of staff reviews. AI can improve the efficiency and quality of the marketing materials reviewed; with human validation of results. Incorporating AI into current review process can reduce the time it takes to review marketing materials, improve efficiency and allow existing FTE resources to be utilized more efficiently, and potentially reduce the volume of marketing misrepresentation complaints. Ideally this will improve efficiency and quality, help beneficiaries access to care, and also ensure marketing materials meets Medicare Marketing Guidelines.",The AI tool required has not been identified as we still need to develop a proof of concept. We anticipate AI would learn the various requirements for Marketing materials to help identify issues and compliance with applicable policies and regulations.,Initiated,Neither,10/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Medicare Part C/D Plan Oversight of AI Used for Prior Authorization and Utilization Management,Department of Health and Human Services,HHS,CMS,Law & Justice,,None of the above.,"Medicare beneficiary access to benefits, services, and prescriptions associated with managed care plans and prescription drug plans. AI tools can increase the capacity to analyze vast data sets based on combined sources. The results can then be further analyzed and refined by SMEs to issue compliance actions, identify plans for program auditing, and help protect the integrity of the Medicare program. ","Evaluate claims, payments, and complaints data to identify outliers. Determine if plans may be out of compliance with CMS rules or are negatively impacting beneficiary health outcomes based on plan use of AI and associated potential bias in system, process, or social determinates of health.",Initiated,Neither,11/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Negotiated Drug Predictions,Department of Health and Human Services,HHS,CMS,Health & Medical,,None of the above.,"The Inflation Reduction Act authorizes Medicare to directly negotiate drug prices for certain high expenditure, single source Medicare Part B or Part D drugs. For the first year of the Negotiation Program, the Secretary will select 10 Part high expenditure, single source drugs for negotiation. Will assist with drug price negotiation for selected drugs",Use historical drug data to predict costs and claim volume for Selected drugs,Initiated,Neither,10/1/2021,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, QPP Admin Bot,Department of Health and Human Services,HHS,CMS,Mission-Enabling (internal agency support),,None of the above.,"Provides custom reporting and analysis to meet the ongoing and emerging needs to address analytics and reporting needs and supports key business processes in the QPP cycle - such as delivering feedback to providers, calculating final scores, and delivering payment adjustments to the Medicare contractors. The Admin Bot would benefit the program by increasing availability of data, thus empowering users and informing decisions, through a simple-to-use communication platform.","Powered by Amazon Bedrock, the out-of-the-box solution can interface directly with data lakes like QPP's Universal Data Set (UDS) empowering users to ask very pointed questions on model data and/or model performance.",Initiated,Neither,8/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Operational Efficiency Analytics ,Department of Health and Human Services,HHS,CMS,Health & Medical,,None of the above.,"OHI/MAG is responsible for providing Marketplace consumers with an accurate, efficient and timely appeals adjudication process. Examples where we can use predictive analytics: * Identifying trends in operations for managing and completing appeals processing * Predicting the number of appeals during open enrollment * Looking for regional or demographic trends in appeal complexity, completion time, and success",Eligibility Appeals Case Management System (EACMS) - Training a model on the data in our future data lake to be able to do predictive analytics.,Initiated,Neither,7/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, ICPG Governance Tool,Department of Health and Human Services,HHS,CMS,Mission-Enabling (internal agency support),,None of the above.,"The business process it will affect involves the processing and analysis of acquisition plans (APs). It will help by streamlining the manual steps involved in validating and classifying acquisition plans, reducing the time and effort required. ",It will automate the processing and analysis of acquisition plans and classify them as IT-related or non-IT.,Implementation and Assessment,Neither,9/1/2021,9/1/2023,,,Developed with contracting resources.,KMP,No,,,No,,No,No,No,,CMS,Documentation is widely available,,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,CEDAR,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Innovation Center Investment Proposal (ICIP) Model Analysis POC,Department of Health and Human Services,HHS,CMS,Mission-Enabling (internal agency support),,None of the above.,"It will affect the process of document analysis, specifically the analysis of ICIP documents. It will help in ICIP document analysis, with the goal of extracting of insights and recommendations from these documents.","It will extract relevant insights and recommendations from the ICIPs, making use of a RAG-based architecture, and present them through a natural language interface. This interface will allow users to interact with the documents and retrieve the information they need easily.",Implementation and Assessment,Neither,8/1/2024,8/1/2024,,,Developed with contracting resources.,PPMS,No,,,No,,No,No,No,,ICIPs,Documentation is widely available,,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,CEDAR,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, OAGM Rate Card,Department of Health and Human Services,HHS,CMS,Mission-Enabling (internal agency support),,None of the above.,"The business process it will affect involves the analysis of historical price history/trends for labor categories in business proposals for the Office of Acquisition and Grants Management (OAGM). It will help OAGM make better decisions on future contracts by providing them with a comprehensive and standardized view of historical price data for labor categories.","It will assist in normalizing labor categories, enabling the OAGM team to easily identify the most relevant labor category despite the variations in naming conventions found in business proposals.",Implementation and Assessment,Neither,9/1/2021,11/1/2022,,,Developed with contracting resources.,SEAS IT,No,,,No,,No,No,No,,CMS,Documentation has been partially completed,,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,CEDAR,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, CCSQ Now Assist for CSM ,Department of Health and Human Services,HHS,CMS,Government Services (includes Benefits and Service Delivery),,None of the above.,"CCSQ Now Assist for CSM (Customer Service Management) is a product inside ServiceNow (SaaS). It integrates Gen AI with CSM, and uses Now LLM (ServiceNow native Large Language Model) to generate contents based on machine learning (ML). Now Assist for CSM helps agents to improve productivity and efficiency and deliver better services. Improve agents' responsiveness and productivity * Quickly get familiar with a case/chat by getting case/chat summarization * Quickly resolve the case by using auto-generated resolution notes.","Agents have quick access to - Case summarization - Chat and agent hand-off summarization - Resolution Notes Generation",Implementation and Assessment,Neither,2/1/2024,5/1/2024,,,Developed with contracting resources.,Not Reported,No,,,No,,No,No,Yes,,Historical ServiceNow case data,Documentation is missing or not available,,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,QSC,Less than 6 months,Yes,,Yes,,No,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, AI Workspace,Department of Health and Human Services,HHS,CMS,Mission-Enabling (internal agency support),,None of the above.,"The business process this affects involves the ability to easily run AI experiments in a safe and secure environment.  It helps by reducing barriers to AI experimentation, supporting strategic AI infrastructure enablement, and providing hands-on experience for teams to learn and understand the potential risks and benefits of AI.","It utilizes data science tooling, large language models, and other approved AWS services such as Sage Maker and Bedrock to provide a secure and supportive environment for AI experimentation and prototype development.",Acquisition and/or Development,Neither,9/1/2024,3/1/2024,,,Developed with contracting resources.,SEAS IT,No,,,,,,,No,,None,Documentation is missing or not available,,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,CEDAR,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Citation Analysis and Survey Assistant (CASA - Nursing Home Survey CMS 2567),Department of Health and Human Services,HHS,CMS,Law & Justice,,None of the above.,"2567 (CASA) enhances the efficiency and effectiveness of monitoring and reviewing nursing home surveys across the US. It enhances how the Quality, Safety, and Oversight Group (QSOG) and Survey Operations Group (SOG) assess how State Survey Agencies (SSAs) are citing nursing home deficiencies, reported on CMS Form 2567, by employing advanced natural language processing and machine learning technologies.  The system would support the Nursing Home Survey CMS Form 2567 survey work allowing QSOG and SOG staff to follow up on cases of under and over citations at the facilities that are flagged by the system and sent to the State Agencies","It will allow for real-time interaction with survey data, facilitating a deeper understanding of trends and individual case specifics. By flagging discrepancies in citations and providing a robust platform for detailed analysis. CASA supports CMS’s mission to ensure high standards of care and accountability in nursing homes",Acquisition and/or Development,Neither,9/1/2023,9/1/2024,,,Developed with both contracting and in-house resources.,N/A,No,,,,,,,Yes,,CMS,Documentation has been partially completed,,,,,,Yes,CDRAP,6-12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Hazards & Threats Awareness Data Processing Automation,Department of Health and Human Services,HHS,CMS,Mission-Enabling (internal agency support),,None of the above.,"EPRO is responsible for developing agency-wide situation reports from various open web and email materials.  More effective prioritization and processing of materials for situational awareness, faster delivery of usable work product to decision-makers agency-wide. The manual process takes 2-3 hours per day, and the AI system will reduce the work to a human quality assurance/inspection step estimated to take approximately 30 minutes.","The AI will scrape known websites and monitor relevant resource mailboxes, use a categorization algorithm, and summarize and format the significant events each morning.",Acquisition and/or Development,Neither,11/1/2024,11/1/2024,,,Developed with both contracting and in-house resources.,SEAS IT,No,,,,,,,Yes,,EPRO,Documentation is widely available,,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Hazard and Threat Awareness,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, Independent Dispute Resolution (IDR) Eligibility Rules Engine,Department of Health and Human Services,HHS,CMS,Mission-Enabling (internal agency support),,None of the above.,"The current Independent Dispute Resolution (IDR) Technical Assistance (TA) process is very manual and time intensive which limits throughput.  The rules engine should significantly expedite processing and expand capacity. Automating more of the process may also increase consistency across recommendations and result in a more predictable timeframe for the workflow by better positioning disputes for analyst review. The AI tool will help automate the eligibility review process, reducing time-intensive manual steps and increasing consistency of results. ","Use of artificial intelligence (AI) models to identify the presence or absence of necessary data points within documentation.  AI tool searches documentation to identify and store necessary data points such as document title, file type, payment date, service code, claim number, and date of service. ",Acquisition and/or Development,Neither,8/1/2024,9/1/2024,,,Developed with contracting resources.,N/A,No,,,,,,,Yes,,Federal Independent Dispute Resolution (IDR) Dispute Data,Documentation is complete,,,,,,Yes,AWS,Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, Medicare Fee for Service Requirements Modernization (MFRM),Department of Health and Human Services,HHS,CMS,Government Services (includes Benefits and Service Delivery),,None of the above.,"EIG is developing a business process to parse, curate, and simplify Medicare coverage and coding requirements by medical condition using AI LLMs.  After rigorous testing for accuracy, AI output will be uploaded to the MFRM application to be used to enhance CMS and healthcare operational efficiencies, including organization of guidance documents, and identification of source complexities to be triaged with owners. ","Utilizing an LLM (Anthropic’s Claude 3.5) to curate current, accurate, and reliable Medicare requirements information sourced from FFS policy and guidance documents.",Acquisition and/or Development,Neither,1/1/2021,1/1/2024,,,Developed with contracting resources.,N/A,No,,,,,,,No,,Medicare Fee-for-Service coverage and coding information,Documentation has been partially completed,,,,,,No,,6-12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Review Regulatory Comments,Department of Health and Human Services,HHS,CMS,Mission-Enabling (internal agency support),,None of the above.,"The Division of Issuer Management and Operations is considering having our contractor, AIR, use an AI tool for comment analysis in the Letter to Issuers (LTI) and Notice of Benefits and Payment Parameters (NBPP). This would streamline review of stakeholder comments. This tool would reduce the amount of time and resources CCIIO and contracting staff would have to review the LTI and NBPP by 25%. ",The AI tool would identify trends in responses from stakeholders in policy and operational issues proposed by CMS. ,Initiated,Neither,12/1/2023,,,,Developed with contracting resources.,Not Reported,No,,,,,,,,,CMS,,,,,,,Yes,Not Reported,,,,,,,,,,,,,,,,,,,,,,,,,,, Improved Data Quality Checks,Department of Health and Human Services,HHS,CMS,Mission-Enabling (internal agency support),,None of the above.,"Improve consumer experience by providing an asynchronous solution to detect and provide near real time feedback via outreach to the consumer shortening the overall return cycle time without requiring UI changes. Improvement of user experience", develop a POC classifier model to identify incorrect document upload types / low-quality images through use of OCR.,Acquisition and/or Development,Neither,7/1/2024,1/1/2024,,,Developed with contracting resources.,N/A,No,,,,,,,No,,CMS,Documentation is widely available,,,,,,Yes,FFM,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Plan Certification Recommendation Model,Department of Health and Human Services,HHS,CMS,Mission-Enabling (internal agency support),,None of the above.,"Identify trends/patterns within plan certification justification templates, containing free-form text data through use of NLP and classification techniques. Improve efficiency of the plan certification review process", build a supervised machine learning model using historical justification data and associated plan certification outcomes that can be recommended to CMS to build towards a more efficient review process.,Acquisition and/or Development,Neither,7/1/2024,1/1/2024,,,Developed with contracting resources.,Not Reported,No,,,,,,,No,,CMS,Documentation is widely available,,,,,,Yes,FFM,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, AI-Assisted Data Entry,Department of Health and Human Services,HHS,HHS,Mission-Enabling (internal agency support),,Inputting large amounts of data from paper forms into a digital system using AI.,"The intended purpose is to leverage AI technology to intake paper forms and assist in the data entry process, which will increase efficiency and reducing human error.",The output is a digital data set converted from physical forms with the assistance of AI tool.,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Transcription in Zoom and Teams,Department of Health and Human Services,HHS,HHS,Mission-Enabling (internal agency support),,Transcribing and summarizing a recorded meeting or interview using AI.,"Meetings scheduled via Zoom or Teams can be transcribed live or from recorded meetings, which improves communication. ",The output is a transcribed text from a live meeting or recorded meetings.,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Email Prioritization in Outlook,Department of Health and Human Services,HHS,HHS,Mission-Enabling (internal agency support),,Prioritizing and categorizing incoming emails using AI.,"The Outlook in Office365 suite can categorize incoming emails based on users' definitions, which can help users manage their inbox more efficiently. The output is categorized emails.",The output is categorized emails.,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, AI-assisted Pattern Detection,Department of Health and Human Services,HHS,HHS,Mission-Enabling (internal agency support),,Identifying unusual patterns in system logs from a single incident report using AI.,"AI technologies have been integrated into many existing log systems to identify unusual patterns, which can help detect system log issues and raise alerts for human intervention. ",The output is alerts and reports that highlight unusual patterns in system logs.,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Report Summarization by commercial GenAI platform,Department of Health and Human Services,HHS,HHS,Mission-Enabling (internal agency support),,Summarizing the key points of a lengthy report using AI.,"Some GenAI platforms, such as ChatGPT and Google Gemini, can summarize documents with non-sensitive information. ",The output is a summary of provided paragraphs or documents.,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Information Search from Google,Department of Health and Human Services,HHS,HHS,Mission-Enabling (internal agency support),,Searching for information using AI.,Google's search engine provides AI-assisted techniques to process search requests efficiently. ,The output is relevant search results.,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Document Digitization by AdobePro,Department of Health and Human Services,HHS,HHS,Mission-Enabling (internal agency support),,Digitizing text from scanned documents or smart forms for archival purposes using AI.,Adobe Pro can convert scanned documents into digital data., The output is digital data with searchable functionality.,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Autocorrect in MS Word,Department of Health and Human Services,HHS,HHS,Mission-Enabling (internal agency support),,Improving the quality of written communications using AI tools.,"MS Word's autocorrect feature uses AI to suggest corrections, which improves the quality of written communications. ",The output is autocorrected words or sentences in written documents.,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, AI-Assisted Real-time Collaboration,Department of Health and Human Services,HHS,HHS,Mission-Enabling (internal agency support),,Collaborating in real-time using AI-assisted tools in word processors.,"The AI-assisted real-time collaboration tool can process the inputs from team members and provide real-times stats using NLP, which offers real-time feedback for collaboration. ",The output includes summarization and statistical data derived from the inputs of multiple users.,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Presentation Design by MS PowerPoint,Department of Health and Human Services,HHS,HHS,Mission-Enabling (internal agency support),,Creating visually appealing presentations using AI-driven design suggestions.,MS PowerPoint can provide more appealing design suggestions based on the content. ,The output is multiple visual design suggestions for the user to select.,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Data modeling and visualization by MS Excel,Department of Health and Human Services,HHS,HHS,Mission-Enabling (internal agency support),,Creating visual representations of data sets for reports and presentations using AI.,"MS Excel contains many data analysis toolkits to provide AI-powered data analysis and visualization, which can improve understanding and presentation of data. ","The outputs from these features are machine learning models and statistical analyses of given data, along with their visual representations.",Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, News from commercial publisher such as Google News,Department of Health and Human Services,HHS,HHS,Mission-Enabling (internal agency support),,Curating news articles and updates based on user preferences using AI.,"Major news publishers, such as Google News, send selective news based on users' preferences and recent visits. ",The outputs from these news providers are more customized news feeds.,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Travel Route Optimization by Google Maps and Apple Maps,Department of Health and Human Services,HHS,HHS,Mission-Enabling (internal agency support),,Planning travel routes using AI-driven map applications.,"Major commercial map service providers, such as Google Maps or Apple Maps, can automate and optimize travel routes based on algorithms and live traffic data, which can save time and provide optimal routes. ",The output from these map services is an optimized travel route based on the user's preferences and live traffic data.,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, AI-Powered Travel Booking,Department of Health and Human Services,HHS,HHS,Mission-Enabling (internal agency support),,Finding and booking travel accommodations using AI-powered platforms.,"Major travel booking websites, such as hotels.com, can provide booking options based on customers' preferences. Algorithms running in such services can optimize travel accommodations, which can save time and provide better deals. ",The output is a list of travel accommodations for customers to choose from and make booking decisions.,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Facial Recognition Unlock,Department of Health and Human Services,HHS,HHS,Mission-Enabling (internal agency support),,Unlocking smartphones or other devices without the need for passwords or PINs using AI-based facial recognition.,The Face ID feature in iPhones takes advantage of AI technology to securely unlock the phone. ,This feature allows the phone to be unlocked using facial recognition technology.,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, AI to Improve Public Access to the Administrative Appeals Process,Department of Health and Human Services,HHS,DAB,Mission-Enabling (internal agency support),,None of the above.,"The DAB adjudicates disputes under more than 60 statutory authorities. The workload is divided between the DAB's four adjudicatory divisions (e.g., the Board, the Civil Remedies Division, the Medicare Operations Division and the Alternative Disputes Resolution Division. Helping appellants decide which entity within the DAB is responsible for adjudicating their appeal will increase efficiency in adjudication and customer experience. The DAB's appellant community (e.g., e-file users) will be provided immediate assistance and accurate information; deaf and hard-of-hearing populations and individuals with limited English proficiency are helped without the use of intermediaries, such as interpreters, translators, and relay services. After determining the appropriate division within the DAB responsible for adjudicating the appeal, the Chatbot can provide a link to the e-file website directly connecting the user to the correct e-file system. Other benefits include: Improve public access to the appeals process by providing instant, accurate and complete filing information; Increase the DAB's productivity by refocusing the workforce to case resolution instead of responding the filing inquires and processing and appropriately routing misfiled appeals; and Improve resource allocation based on each Division's actual workload. ","A chatbot on the DAB's website can direct filers to the correct Division and e-file system with which the appeal should be filed by using natural language processing (e.g., questions and responses) with the ability to further refine and increase accuracy based on user response. ",Acquisition and/or Development,Neither,4/1/2023,7/1/2023,,,Developed in-house.,,,,,,,,,Yes,,None,Documentation is complete,,,,,,Yes,DAB E-file,More than 12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, AI Use Policy Tool,Department of Health and Human Services,HHS,DAB,Mission-Enabling (internal agency support),,None of the above.,"The DAB is responsible for issuing fair, impartial, legally correct and defensible determinations which serve as the final decision of the DHHS Secretary. The AI Use Policy is designed to promote public confidence in the integrity of DAB decisions. More effective quality review tool and faster identification of data trends that require additional analysis. ","AI Use Policy Tool will scan DAB decisions to ensure compliance with quality review standards (e.g., protect PHI, PII and FTI) and identify work product that potentially violates the prohibition of the use of generative AI in DAB decisions. ",Acquisition and/or Development,Neither,4/1/2023,9/1/2023,,,Developed in-house.,,,,,,,,,Yes,,TBD,Documentation is complete,,,,,,Yes,DABACTS,More than 12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, Resources to Assist the Advisory Board In Identifying AI Tools for Use In An Adjudication Environment,Department of Health and Human Services,HHS,DAB,Mission-Enabling (internal agency support),,None of the above.,"The adjudication process from docket analysis, case processing, issuing a decision and quality review (after adjudication) relies on the analysis of large quantities of data. AI or machine learning tools will help the Advisory Board determine which technology will best serve the Agency's needs in an adjudication environment. ",The use of AI to analyze the voluminous data received from appellants and interested parties to identify trends and increase efficiency in the adjudication process. ,Acquisition and/or Development,Neither,4/1/2023,9/1/2023,,,Developed in-house.,,,,,,,,,No,,None,Documentation is complete,,,,,,No,,More than 12 months,No,,No,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, 356H ML Facility Supply Chain Role Classification,Department of Health and Human Services,HHS,FDA,Mission-Enabling (internal agency support),,None of the above.,"Applicants submit unstructured text to indicate a facility’s supply chain role, which range from manufacturing, testing, storing, packaging, etc. This use case extracts the supply chain role based on the text and presents it to data stewards who perform data operations Improve the efficiency of evaluating 356H forms to assess a facility's supply chain role, thereby decreasing the time required for the processing of submissions.",Extracts a facility's supply chain role from industry submissions and displays identified roles on a data-stewards screen for human verification and final determination.,Operation and Maintenance,Neither,6/1/2022,6/1/2022,5/1/2023,,Developed with both contracting and in-house resources.,Not Reported,,,,No,,No,No,No,,Applicant submissions,Documentation is widely available,,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,CDEROne Analytics: FISMA-high environment,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, AI Tool for Risk-based FAR Review & Decision Support,Department of Health and Human Services,HHS,FDA,Mission-Enabling (internal agency support),,None of the above.,"Assist Field Alert Report (FAR) reviewers by providing risk-based intelligence and insights for review and decision support with a Human-in-the-Loop. Assist with streamlining an efficient FAR review process by summarizing risk profiles for human exploration that is evaluated with additional data to assist human decisioning.","AI-based machine learning classification of FAR risk into low, medium, and high; provides insights on problem clusters, rare-events, and source variables.",Operation and Maintenance,Neither,6/1/2020,10/1/2020,10/1/2024,,Developed in-house.,,,,,No,,No,No,Yes,,Internal Field Alert Reports data from LSMV,Documentation has been partially completed,,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,CDEROne Analytics: FISMA-high environment,6-12 months,Yes,,Yes,,No,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Analytics-Driven Supplement Evaluation (ASE),Department of Health and Human Services,HHS,FDA,Mission-Enabling (internal agency support),,None of the above.,"This AI use case supports the triage and staff assignment process for the review of post-market Change Being Effected (CBE) supplement submissions. The output of the AI component contributes to the triage and staff assignment process, and summarizes submitted information to support the decision of the reviewers of the CBE 30/0 supplemental submissions.","Using a Convolutional Neural Network (NN) model, in combination with a rules-based approach, produces an output that helps staff triage CBE submission review",Acquisition and/or Development,Neither,10/1/2022,10/1/2022,,,Developed in-house.,,,,,,,,,No,,Data submitted in applicants' supplemental submissions,Documentation is complete,,,,,,No,,6-12 months,No,,No,,No,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Augmenting death and cause of death ascertainment in observational data sources,Department of Health and Human Services,HHS,FDA,Health & Medical,,None of the above.,"Augment assessment of mortality in electronic health records to support investigations to develop probabilistic estimates for causes of death in the Sentinel system. The extraction of mortality information improves studies in Sentinel using mortality as an endpoint by enhancing on the quality of mortality data.","Augment assessment of mortality (date of death, cause of death) by applying Natural Language Processing (NLP) on healthcare narrative text and administrative codes.",Operation and Maintenance,Neither,1/1/2022,1/1/2022,12/1/2023,,Developed with contracting resources.,75F40119D10037 - 7540119F19002,,,,No,,No,No,Yes,,"`- National Death Index (NDI) data - State Death Certificate Registery",Documentation has been partially completed,,,Yes,No – agency does not have access to source code.,,Yes,Sentinel,6-12 months,No,,No,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, CLAT (Computerized Labeling Assessment Tool) ,Department of Health and Human Services,HHS,FDA,Mission-Enabling (internal agency support),,None of the above.,"CLAT uses computer programming, image processing, and AI to assist the human review of drug labeling, including container labels, carton labeling, prescribing information, and Instructions for Use. CLAT assists the human visual comparison of drug labeling against a library of FDA regulations, guidances, standards, and best practices. CLAT identifies potential labeling deviations for human evaluation. CLAT is used to improve efficiencies in the manual labeling review process to help the reviewer mitigate medication errors and standardize labeling across therapeutic drug classes and product types. CLAT also assists the reviewer to catch deviations between label versions so no unintended errors have been introduced.","The tool processes images of carton and container labeling to identify minimum requirements on a label, identify availability of objects on an image, color differentiation of strength, missing barcode and orientation, incorrect or missing strength statement, error prone abbreviation, look-alike labels and text prominence. This will assist the reviewer by improving the efficiency of drug labeling reviews, including container labels, carton labeling, prescribing information, and Instructions for Use. ",Acquisition and/or Development,Neither,9/1/2020,9/1/2020,,,Developed with contracting resources.,N/A,,,,,,,,Yes,,Label and Container cartons,Documentation is complete,,,,,,Yes,CDEROne Analytics: FISMA-high environment,6-12 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Empirical evaluation of EHR-based signal detection approaches,Department of Health and Human Services,HHS,FDA,Health & Medical,,None of the above.,"This supports Electronic Health Record (EHR) signal detection by extracting information using Natural Language Processing (NLP) and expanding the tree based scan statistic (TBSS) methods for EHR-based signal detection. Extracts data to facilitate the identification of signals for outcomes, improving the manual identification of signals for outcomes derived from electronic health records.",The AI component in this study leverages an Natural Language Processing (NLP) approach to extract data from unstructured clinical notes.,Acquisition and/or Development,Neither,9/1/2023,9/1/2023,,,Developed with contracting resources.,75F40119D10037 - 75F40123F19010,,,,,,,,Yes,,Claims-Electronic Health Record (EHR),Documentation has been partially completed,,,,,,Yes,Sentinel ,Less than 6 months,No,,No,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, FAR-based Facility Signal Detection Tool ,Department of Health and Human Services,HHS,FDA,Mission-Enabling (internal agency support),,None of the above.,"Identifies problem clusters for the flagged signals in an objective manner for triage/review. Assist with objectively identifying problem clusters for the flagged signals, which then go through a dedicated triage & review process for further actions, and if needed, CMS (inspection) work activities can be opened. ",Identifies problem clusters for the flagged signals in an objective manner for triage/review. ,Operation and Maintenance,Neither,6/1/2019,6/1/2020,6/1/2024,,Developed in-house.,,,,,No,,No,No,Yes,,Internal Field Alert Reports data from LSMV (FAERS tool),Documentation has been partially completed,,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,CDEROne Analytics: FISMA-high environment,6-12 months,Yes,,Yes,,No,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, MedWatch Dashboard ,Department of Health and Human Services,HHS,FDA,Mission-Enabling (internal agency support),,None of the above.,"Assist with consistent monitoring and identification of product risks from MedWatch reporting patterns and report content to support review staff. Supports efficient, objective, risk-based product quality assessments from MedWatch reports for verification by review staff ",Identify product risk signals from MedWatch reports by using time series analysis to flag products and topic modeling to summarize the comments.,Operation and Maintenance,Neither,1/1/2021,5/1/2021,4/1/2023,,Developed in-house.,,,,,No,,No,No,Yes,,MedWatch data from LSMV and IQVIA data,Documentation is complete,,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,CDEROne Analytics: FISMA-high environment,6-12 months,Yes,,Yes,,No,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Quality Surveillance Dashboard (QSD),Department of Health and Human Services,HHS,FDA,Mission-Enabling (internal agency support),,None of the above.,"Extracts unstructured text from documents and pools that with other available data to form a dashboard that enables consistent assessment of CDER regulated manufacturing facilities Assists in the efficient, objective, risk-based assessments of CDER regulated manufacturing facilities.",Identifies and extracts keywords/phrases from unstructured documents and presents sentences containing keywords/phrases in context.,Operation and Maintenance,Neither,1/1/2019,1/1/2019,3/1/2023,,Developed in-house.,,,,,No,,No,No,Yes,,FDA EIR documents,Documentation is complete,,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,CDEROne Analytics: FISMA-high environment,6-12 months,Yes,,Yes,,No,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Scalable automated NLP-assisted chart abstraction and feature extraction tool,Department of Health and Human Services,HHS,FDA,Mission-Enabling (internal agency support),,None of the above.,"Examine the usability of electronic medical records in conducting pharmacoepidemiology studies through a case study of montelukast use among patients with asthma and neuropsychiatric events The project will involve an “end-to-end” implementation of a full pharmacoepidemiologic study using one or more product-outcome pairs to demonstrate how NLP and other advanced analytic methods or tools can be used to accurately identify study outcomes and covariates from unstructured Electronic Health Record (EHR) data.","Natural language processing (NLP), a computational linguistics technology, will be used to read, interpret and organize health data that may be contained within unstructured data fields within EHRs. This activity intends to build a semi-automated system that uses advanced methods and tools, including NLP, to enable large-scale outcome identification, confounder feature extraction, and longitudinal contextualization of Electronic Health Record (EHR) data.",Acquisition and/or Development,Neither,10/1/2022,9/1/2022,,,Developed with contracting resources.,75F40119D10037 - 7540119F19002,,,,,,,,Yes,,Claims-Electronic Health Record (EHR) - (structured and unstructured),Documentation has been partially completed,,,,,,Yes,Sentinel ,6-12 months,No,,No,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Annual Report CMC,Department of Health and Human Services,HHS,FDA,Mission-Enabling (internal agency support),,None of the above.,"Objective of this use case was to extract CMC changes reported within unstructured annual report industry submission documents to build a complete repository for downstream analysis The extracted data are presented in a dashboard (along with other data) for downstream human analysis.",Support information extraction from unstructured documents,Operation and Maintenance,Neither,1/1/2024,1/1/2024,5/1/2024,,Developed with both contracting and in-house resources.,75F40124A00003,,,,No,,No,No,No,,Applicant submissions,Documentation is widely available,,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,CDEROne Analytics: FISMA-high environment,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Application-DMF Reference,Department of Health and Human Services,HHS,FDA,Mission-Enabling (internal agency support),,None of the above.,"Extracts references to DMFs from marketing application (ANDA/NDA/BLA) submissions. These submission documents may be structured (356H form) or unstructured (eCTD module 1-4). This pipeline parses content from these documents, extracts DMF references (e.g., ANDA123456 references DMF123456), and exposes the data in a structured format for analysis. The extracted data are presented in a dashboard, along with additional data, for downstream human analysis. The purpose of this use case is to build a comprehensive inventory of all application to DMF references to support OPQ in reviewing applications since many application to DMF references were previously missing from structured sources.",Support information extraction from unstructured documents,Operation and Maintenance,Neither,5/1/2023,5/1/2023,12/1/2023,,Developed with both contracting and in-house resources.,75F40124A00003,,,,No,,No,No,No,,applicant submissions,Documentation is widely available,,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,CDEROne Analytics: FISMA-high environment,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Artificial Intelligence-based Deduplication Algorithm for Classification of Duplicate Reports in the FDA Adverse Event Reports (FAERS) ,Department of Health and Human Services,HHS,FDA,Mission-Enabling (internal agency support),,None of the above.,"The deduplication algorithm is applied to all historical and incoming case report submissions to the FDA Adverse Event Reporting System (FAERS) to identify duplicate individual case safety reports (ICSRs). Unstructured data that is in free text FAERS narratives is processed through a natural language processing system to extract relevant clinical features. Both structured and unstructured data are then used in a probabilistic record linkage approach to identify duplicates. Application of the deduplication algorithm is optimized for processing entire FAERS database to support the data mining analytics process. Enhance the efficiency of removing duplicate FAERS reports to support data mining.",Uses Natural Language Processing (NLP) to extract out clinical features from free text narratives in a case safety report and uses them in the identification of duplicative case safety reports in the FAERS database.,Implementation and Assessment,Neither,12/1/2023,9/1/2024,,,Developed with contracting resources.,BAAFY23C2AWP1,,,,No,,Yes,No,Yes,,Adverse Event data in FAERS,Documentation is complete,Sex/Gender,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,6-12 months,Yes,,Yes,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Development of automation tools and data warehouse to facilitate BE assessments,Department of Health and Human Services,HHS,FDA,Mission-Enabling (internal agency support),,None of the above.,"Enhancing and streamlining the data entry associated with the review of bioequivalence data in Abbreviated New Drug Application (ANDA) submissions Reduce manual effort by preparing data for analysis and exporting results as part of the bioequivalence review. This tool will facilitate data collection and Information retrieval from the firm-submitted ANDA packages but will not involve reviewers' evaluation or judgement.",Transform submitted PK data into a usable format so that the reviewer can perform the routine bioequivalence analysis using the authorized pre-written SAS codes. Then it exports results to a draft bioequivalence review document with the firm-submitted eCTD tables.,Acquisition and/or Development,Neither,5/1/2018,5/1/2018,,,Developed with both contracting and in-house resources.,75F40120F80605,,,,,,,,No,,N/A,Documentation is widely available,,,,,,Yes,CDEROne Analytics: FISMA-high environment,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, DMF (Drug Master File) Facilites,Department of Health and Human Services,HHS,FDA,Mission-Enabling (internal agency support),,None of the above.,"Extracts facility references from Type II (Drug Substance) DMF manufacturing submissions (i.e., DMF123456 discloses that it uses Facility X for manufacturing and Facility Y for stability testing). These DMF submissions may include structured documents (3938 form) or unstructured documents (eCTD module 3). This pipeline parses content from these documents, extracts facility references and supply chain roles (i.e., FEI 6789 – manufacturing facility), and exposes the data in a structured format for analysis. This allows OPQ to identify discrepancies between facilities reported on marketing applications and those disclosed by DMFs referenced by those applications. The extracted data are presented in a dashboard, along with additional data, for downstream human analysis. The ultimate purpose of this use case is to build a comprehensive inventory of facilities reported within DMF submissions as well as the applications that reference those DMFs.",Support information extraction from unstructured documents,Operation and Maintenance,Neither,6/1/2022,6/1/2022,6/1/2023,,Developed with both contracting and in-house resources.,75F40124A00003,,,,No,,No,No,No,,Applicant submissions,Documentation is widely available,,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,CDEROne Analytics: FISMA-high environment,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Four Part Harmony Identification,Department of Health and Human Services,HHS,FDA,Mission-Enabling (internal agency support),,None of the above.,"Accuracy of language used to report back deficiencies to submitters. This will reduce the amount of time it takes and reduce staff burden, thus allowing for MDUFA requirements to be met.",This will reduce the amount of time it takes to verify and identify issues with language used for deficiency letters and ensure the adherence to the language necessary.,Acquisition and/or Development,Neither,9/1/2023,9/1/2023,,,Developed in-house.,,,,,,,,,Yes,,Submission data and documents,Documentation is widely available,,,,,,Yes,CEDh,6-12 months,Yes,,Yes,,Yes,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, Information Visualization Platform (InfoViP) to Support Analysis of adverse event reports,Department of Health and Human Services,HHS,FDA,Mission-Enabling (internal agency support),,None of the above.,"Information Visualization Platform (InfoVIP) supports post-market surveillance using AI to assist prioritization and analysis of adverse event reports by extracting data from unstructured case narratives and combining it with structured data to support analysis and review of adverse event reports. The tool assists with case series analyses, creating data visualization, and classifying adverse event reports by information quality. Improve the efficiency in preparing and analyzing data during the review and evaluation of adverse event reports.",Performs Natural Language Processing (NLP) and applying Machine Learning (ML) algorithm to extract data from unstructured case narratives and combine with structured data to support analysis and review of adverse event reports.,Acquisition and/or Development,Neither,9/1/2019,9/1/2019,,,Developed with contracting resources.,BAAFY23C2AWP1,,,,,,,,Yes,,Adverse Event data in FAERS,Documentation has been partially completed,Sex/Gender,,,,,No,,More than 12 months,Yes,,Yes,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, LLM-Assisted VAERS Analyses,Department of Health and Human Services,HHS,FDA,Health & Medical,,None of the above.,"Build capacity for and assess the application of a LLM to VAERS (Vaccine Adverse Events Reporting System) to provide reviewers adhoc VAERS queries and efficiently generate customized query outputs Efficiently generate customized query outputs of VAERS queries for reviewers.",To provide reviewers adhoc VAERS queries.,Acquisition and/or Development,Neither,12/1/2023,12/1/2023,,,Developed in-house.,,,,,,,,,Yes,,Adverse Events data in VAERS,Documentation is widely available,"Sex/Gender, Age, Race/Ethnicity ",,,,,Yes,CBER HIVE,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Pharmacovigilance machine learning to detect excess safety signal,Department of Health and Human Services,HHS,FDA,Government Services (includes Benefits and Service Delivery),,None of the above.,"Post-market adverse event reporting Using a computer algorithm assists FDA in finding ""needle in the haystack"" events that are difficult to uncover with manual review of reports",Generate potential safety signals to investigate further using statistical based algorithms,Operation and Maintenance,Neither,1/1/2015,1/1/2008,1/1/2015,,Developed with contracting resources.,N/A,,,,No,,No,No,No,,N/A,,,,Yes,,,Yes,PV Analyzer,,No,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Module 3 Facilties,Department of Health and Human Services,HHS,FDA,Mission-Enabling (internal agency support),,None of the above.,"Objective of this use case was to identify and extract all drug manufacturing facilities reported within unstructured module 3 submissions from marketing applications to build an comprehensive inventory of drug facilities The extracted data are presented in a dashboard, along with additional data, for downstream human analysis.",Support information extraction from unstructured documents,Operation and Maintenance,Neither,7/1/2023,7/1/2023,2/1/2024,,Developed with both contracting and in-house resources.,75F40124A00003,,,,No,,No,No,No,,applicant submissions,Documentation is widely available,,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,CDEROne Analytics: FISMA-high environment,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, OPPQ Policy Bot (AKA Policy and Guidance Documents Search) ,Department of Health and Human Services,HHS,FDA,Mission-Enabling (internal agency support),,None of the above.,"Helps policy staff in searching documents containing regulations and responding to inquiries, and other policy related work. In developing policy and responding to inquiries, OPPQ staff often need to reference previously published policy documents in order to ensure consistency. This AI-enabled tool will allow policy leads and other staff to quickly identify and query the appropriate documents.","Identifies applicable published MAPPS, guidances, and policy related documents based on queries from users. Includes link to the document for further human review.",Implementation and Assessment,Neither,9/1/2024,9/1/2024,,,Developed with both contracting and in-house resources.,75F40124A00003,,,,No,,No,No,No,,CDER policy documents,Documentation is widely available,,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,CDEROne Analytics: FISMA-high environment,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Packaging Materials and Suppliers,Department of Health and Human Services,HHS,FDA,Mission-Enabling (internal agency support),,None of the above.,"Objective of this use case aims is to extract data from unstructured sources to build an inventory of drug packaging materials and their suppliers to support staff on conducting drug supply chain analysis The extracted data are presented in a dashboard for downstream analysis.",Support information extraction from unstructured documents,Operation and Maintenance,Neither,7/1/2023,7/1/2023,9/1/2024,,Developed with both contracting and in-house resources.,75F40124A00003,,,,No,,No,No,No,,Applicant submissions,Documentation is widely available,,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,CDEROne Analytics: FISMA-high environment,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Process Large Amount of Submitted Docket Comments,Department of Health and Human Services,HHS,FDA,Mission-Enabling (internal agency support),,None of the above.,"To enhance the automatic process of dockets, we have created an AI/ML tool in CBER/HIVE that automatically download dockets and process them to accelerate the review of docket comments, significantly improving the efficiency and accuracy of our regulatory processes Efficiently generate customized query outputs of VAERS queries for reviewers.",To provide reviewers adhoc VAERS queries.,Implementation and Assessment,Neither,9/1/2021,11/1/2021,,,Developed in-house.,,,,,No,,No,No,No,,Public comments for various dockets.,Documentation is widely available,,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,CBER HIVE,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Real World Data/Evidence ,Department of Health and Human Services,HHS,FDA,Mission-Enabling (internal agency support),,None of the above.,"Identify industry unstructured submissions containing Real World Data/Evidence (RWD/E) by analyzing parsed content for likely indicators of Real World Data/Evidence RWD/E to support congressional reporting. The extracted data are presented in a dashboard for downstream human analysis.",Supports extracting text from unstructured documents and tagging for documents containing Real World Evidence and Real World Data,Implementation and Assessment,Neither,2/1/2024,2/1/2024,,,Developed with both contracting and in-house resources.,75F40124A00003,,,,No,,No,No,No,,applicant submissions,Documentation is widely available,,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,CDEROne Analytics: FISMA-high environment,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Regulatory Starting Material ,Department of Health and Human Services,HHS,FDA,Mission-Enabling (internal agency support),,None of the above.,"Extract RSMs and their suppliers from unstructured module 3 industry submissions to create an inventory that will illuminate the upstream supply chain. The extracted data are presented in a dashboard for downstream analysis.",Support information extraction from unstructured documents,Operation and Maintenance,Neither,9/1/2022,9/1/2022,8/1/2023,,Developed with both contracting and in-house resources.,75F40124A00003,,,,No,,No,No,No,,applicant submissions,Documentation is widely available,,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,CDEROne Analytics: FISMA-high environment,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Resource Capacity Planning,Department of Health and Human Services,HHS,FDA,Mission-Enabling (internal agency support),,None of the above.,"Forecasting human drug review program submissions and corresponding FDA workload Improve the forecasting of human drug review program expected workload in order to adjust fee revenue and fees to fund resources needed to complete expected workload ",Forecasts workload submissions across major user fee programs to support fee setting for the human drug review programs,Operation and Maintenance,Neither,1/1/2016,1/1/2017,8/1/2020,,Developed with both contracting and in-house resources.,HHSF223201510012B,,,,No,,No,No,Yes,,FDA systems including DARRTS and Panorama,Documentation is widely available,,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,CDEROne Analytics: FISMA-high environment,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, Supply Chain Resilience Program,Department of Health and Human Services,HHS,FDA,Emergency Management,,None of the above.,"Forecasting demand of medical devices and supplies. Forecast demand for critical devices during a variety of scenarios (e.g. natural disaster, PHE) ",Aids in forecasting demand for critical devices under a variety of scenarios. ,Operation and Maintenance,Neither,2/1/2022,2/1/2022,4/1/2023,,Developed with contracting resources.,N/A,,,,No,,No,No,Yes,,N/A,Documentation is widely available,,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,FiDLE; RSCP and is being moved to CEDh-OSCR,6-12 months,,,Yes,,Yes,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, AI Audit Resolution Assistant,Department of Health and Human Services,HHS,HRSA,Mission-Enabling (internal agency support),,None of the above.,"The objective of this initiative is to reduce the time for HRSA auditors to review/resolve Single audits. HRSA has experienced an exponential increase in Single audits which is attributable in COVID funding attributable to the Provider Relief Fund programs. Success will be measured through the increase in the number of Single audits assigned to each auditor. e.g., improved technological capabilities should allow HRSA Auditors to more efficiently review and closeout Single audits. ","HRSA will utilize generative AI to streamline the Single Audit resolution process and the creation of Management Decision Letters to formally close out and resolve Single audit findings. The AI Audit Resolution Assistant (AIARA) includes a vector database comprised of Single audit documents assigned to HRSA, creates a retrieval augmented generation which integrates with a large language model to intelligently summarize audit findings and recommendations, and provides chatbot capability and reduce the cognitive load on HRSA auditors for any audit-specific questions. ",Implementation and Assessment,Neither,1/1/2024,3/1/2024,,,Developed with both contracting and in-house resources.,75R60222F80097,,,,No,,No,No,No,,HRSA did not use agency data to train or fine-tune the large language model. HRSA did use publicly-available data from the Federal Audit Clearinghouse to evaluate the accuracy and precision of LLM outputs. ,Documentation is widely available,,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,"At the time of this response, HRSA is evaluating multiple HHS infrastructuure hosting services for Azure OpenAI platform with ATO in place. A decision on a specific environment is forthcoming.",Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, Knowledge Navigator,Department of Health and Human Services,HHS,HRSA,Government Services (includes Benefits and Service Delivery),,None of the above.,"The objective is to develop an AI model that can answer detailed and complex questions about the key programmatic document, Application and Program Guidance (APG), that is issued annually before the application cycle opens. The PoC LLM has Applicant and Program Guidance (APG) documents for 10 loan repayments and scholarship programs. This will allow loan repayment and scholarship analysts and call center agents to better respond to public inquiries from applicants and participants. If successful the Knowledge Navigator could be made available to applicants and participants and reduce the volume of calls to the BHW Call Center along with the number of call center agents. ",BHW has implemented a proof of concept Generative AI (GenAI) Large Language Model (LLM) Knowledge Navigator (KN) to support National Health Service Corps (NHSC) and Nurse Corps loan repayment and scholarship program analysts and call center agents respond to program applicants and participants. ,Acquisition and/or Development,Neither,2/1/2024,2/1/2024,,,Developed with both contracting and in-house resources.,75R60224F34005,,,,,,,,No,,HRSA Application and Program Guidance (APG) documents for 10 HRSA loan repayment and scholarship programs. All of which are public documents. Here is an example: https://nhsc.hrsa.gov/sites/default/files/nhsc/loan-repayment/nhsc-students2service-lrp-application-program-guidance.pdf,Documentation is widely available,,,,,,Yes,Bureau of Health Workforce Management Information System Solution (BMISS) Platform (UII: 009-000004158,Less than 6 months,No,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Policy Assistant,Department of Health and Human Services,HHS,HRSA,Mission-Enabling (internal agency support),,None of the above.,"HRSA is looking forward to leveraging Generative AI tools to support and enhance the current drafting of several policy documents, funding opportunities, and notices. This could streamline the policy development process, reduce drafting time, and improve document quality by providing critical analysis and suggestions for improvement, thus enhancing the overall policy-making workflow. Reduce the amount of time required to develop first drafts of key BHW policy documents including APGs, NOFOs, budget documents","The Policy Assistant (PA) is an innovative application utilizing large language models (LLMs) to generate first drafts of key policy documents. Inputs can include example documents, style guides, and key policy decisions and other documents in the Knowledge Navigator. PA also features an editing tool that scrutinizes drafts for inconsistencies or errors, offering feedback for refinement. Current planning leverages Cloud-based services, LLM, Text processing and analysis tools, Natural Language Generation (NLG) and Text Analysis for the implementation.",Initiated,Neither,3/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Scholar Match,Department of Health and Human Services,HHS,HRSA,Government Services (includes Benefits and Service Delivery),,None of the above.,"The current candidate evaluations and placement process is complex and challenging for both analysts and participants. By enhancing this process with AI/ML support could optimize resource allocation and candidate satisfaction, significantly impacting workforce distribution and efficiency in critical health areas. This would improve the process of matching NHSC and Nurse Corps Scholars going into clinical service in underserved communities."," The Scholar Match (SM) leverages AI to enhance the placement process of NHSC and Nurse Corps scholars in communities of need across the U.S. and territories. By analyzing candidate profiles and regional needs, SM recommends optimal placements, ensuring both the fulfillment of organizational needs and the satisfaction of the candidates. Current planning leverages Machine Learning, Recommendation Systems, Cloud-based platforms and Data analytics services for the implementation.",Initiated,Neither,1/1/2025,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Scholarship Insight,Department of Health and Human Services,HHS,HRSA,Government Services (includes Benefits and Service Delivery),,None of the above.,"The NHSC and Nurse Corps Scholarship reviewers need to receive and score thousands of essays submitted and the manual process delays the application processes. We are looking forward to building a Generative AI program that looks at all of the previous essays and how the scoring rubric was done and train it to do the first cut at scoring the essays. This could enhance the fairness and efficiency of scholarship evaluations, ensuring a thorough review process that supports equitable student opportunities. Reduce the amount of time required for internal or external reviews to evaluated NHSC and Nurse Corps scholarship applications and ensure that human reviewers are following the appropriate scoring rubric.","The Scholarship Insight (SI) is designed to support the evaluation of scholarship essays for both the NHSC and Nurse Corps Scholarship Programs by providing detailed analysis to human graders. Aligning with directives to ensure human oversight, SI identifies key themes, strengths, and weaknesses in essays, facilitating a more informed grading process without replacing human judgment. Current planning leverages Cloud-based services, Natural Language Understanding (NLU), Text Analysis, LLM API and Data analysis tools for the implementation.",Initiated,"Rights-Impacting ",1/1/2025,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Site Application Analysis,Department of Health and Human Services,HHS,HRSA,Government Services (includes Benefits and Service Delivery),,None of the above.,"The current application review process requires internal analysis to review many complex documents to determine Site Eligibility to employ a member of the NHSC or Nurse Corps. By enhancing this process with AI/ML support could revolutionize and streamline the NHSC, Substance Use Disorder Treatment and Recovery (STAR)and Nurse Corps Site Application Process. This would improve the current highly manual process for reviewing Site applications for internal analysts but ensure that clinical sites are being approved for NHSC, Nurse Corps, and STAR LRP.","The Site Application Analysis (SA) is designed to support the evaluation of NHSC, STAR and Nurse Corps Site Applications. SA will allow for faster more accurate review of Site Applications and allow BHW Regional Analysts to focus on a higher value task. Current planning leverages Cloud-based services, Machine Learning, Recommendation Systems, Natural Language Understanding (NLU) and Text Analysis for the implementation.",Initiated,"Rights-Impacting ",1/1/2025,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, A Cluster-Based Machine Learning Approach for Evaluating Factors Associated with Plasma Neurodegenerative Biomarkers,Department of Health and Human Services,HHS,NIH,Health & Medical,,None of the above.,"This AI Use Case is Retired, No Longer in Use. This study investigated whether incorporating health-related risk factors can improve the predictive power of plasma biomarkers (related to neurodegeneration and inflammation) for incident dementia.  Improve incident dementia prediction using plasma neurodegeneration biomarkers in a diverse population. ","Using ~450 common health-related risk factors and plasma neurodegeneration and inflammation biomarkers from the population-based cohort with the follow-up period over 20 years - the Age, Gene/Environment Susceptibility – Reykjavik Study.   ",Retired,Neither,,,,9/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, AccessGUDID Data Validation,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"The artificial intelligence (AI) system for AccessGUDID will focus on enhancing data quality assurance by detecting anomalies in medical device metrics, such as inconsistent units of measurement or unusual device dimensions. It will analyze both structured data, like numerical measurements, and unstructured data, such as free text entries. The AI will not modify data directly but will flag potential issues for human review. Integrated into the U.S. Food and Drug Administration (FDA)'s processing workflow, it will leverage standard codes for precise grouping and employ advanced language models to process unstructured data. Deployed within the regulatory compliance and data management areas of NLM, this AI system aims to streamline data validation processes, reduce manual workload, and maintain the integrity of the AccessGUDID database, without overstepping into areas requiring human oversight. The key problem the AI aims to address is the presence of inconsistent or incorrect data entries, such as mismatched units of measurement or outlier device dimensions, which can lead to regulatory compliance issues and potential risks in device usage. By automating the anomaly detection process, the AI will improve the efficiency of data validation, significantly reducing the manual effort required to identify and correct errors. This will not only enhance the accuracy of the data but also speed up the processing and integration of new submissions into the AccessGUDID database. Improved data integrity will lead to better decision-making for manufacturers, healthcare providers, and regulators, ensuring that medical devices on the market meet safety and performance standards. Ultimately, the positive outcomes of this AI application include increased trust in the medical device information provided to end users, which contributes to the broader goal of improving public health and ensuring patient safety across the nation.","For inputs, it handles structured data, such as numerical metrics (e.g., device dimensions, units of measurement), and unstructured data, such as free-hand text entries describing medical devices. This data is sourced from the AccessGUDID database, which aggregates information submitted by manufacturers to the FDA. The frequency of data processing aligns with the batch submissions from the FDA, typically occurring daily or whenever new data is received. The system processes large volumes of data to detect anomalies, making it scalable to accommodate growing datasets. For outputs, the AI generates predictions and classifications, specifically identifying potential anomalies in the data, such as incorrect units of measurement or outlier device specifications. The results are presented as flags or labels indicating the presence of an anomaly, which are then reviewed by human analysts. The AI produces these results in real-time as part of the batch processing workflow, ensuring that each data submission is evaluated promptly before being fully integrated into the AccessGUDID database. The output helps in streamlining the data validation process and improving overall data quality.",Initiated,Neither,3/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, AI Solution for Image Duplication Detection,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"NIA researchers generate a significant number of microscopic images for each publication, ranging from 100 to 200 TIFF files per study. Ensuring that these images are not duplicated or overlapped in any publications is crucial for maintaining research quality and credibility. The current manual verification process is time-consuming, prone to human error, and insufficient for handling large volumes of data.  NIA researchers generate a significant number of microscopic images for each publication, ranging from 100 to 200 TIFF files per study. Ensuring that these images are not duplicated or overlapped in any publications is crucial for maintaining research quality and credibility.",Identification of image duplications.,Initiated,Neither,5/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, AI Solution for Research Funding Gap Analysis (RF-GAP),Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"NIA is focusing on identifying and addressing funding and publication gaps in NIH-funded research. Using text-mining and artificial intelligence techniques on data from both NIH and external sources, NIA aims to uncover research areas that need more funding or lack sufficient publication coverage. This approach will help NIA leadership make informed decisions, refine funding strategies, and enhance the overall impact of NIH-funded research. Program officers will utilize a user-friendly dashboard for actionable insights, simplifying the tracking of research outputs and identification of gaps.  1. Improved allocation of NIH funding through targeted identification of underfunded research areas and initiatives to address publication gaps. 2. Enhanced integration of data sources will provide a comprehensive view of research trends. AI-driven analysis will offer actionable insights into emerging research areas and future funding needs.  3. Customized dashboards will empower program officers with efficient tools to monitor and manage research portfolios, fostering transparency and maximizing operational efficiency across NIA.  This approach will help NIA leadership make informed decisions, refine funding strategies, and enhance the overall impact of NIH-funded research.",Publication and Gap analysis reports.  ,Initiated,Neither,5/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, AI/ML for Study Section Prediction in NICHD,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"The project of AI/ML for Study Section Prediction is to classify the grant applications to specific Study Sections in the Scientific Review Branch in NICHD.  It employs human-in-the-loop development which enables human evaluation and offers the ability to override and alter model outputs.  Source data are from the NIH IMPAC II system via the eRA Review Module. The AI process and output data have no impact on the NIH IMPAC II system. The business area addressed is the alignment of appropriate Study Section for a given grant application to assist the Scientific Review Branch (SRB) at NICHD. This project adapts the methods and systems developed for the RPAB AI/ML Referral System project to optimize the assignment of grant applications to the most suitable study sections. The AI system will contribute to NIH's mission by improving the efficiency and accuracy of alignment of grant applications to study sections, ensuring that applications are reviewed by the most appropriate researchers, leading to scientific discoveries and health advancement.",Internal: NIH ImpacII and eRA Review Module queries. Results are presented as study section (class) predictions and class probabilities.,Implementation and Assessment,Neither,10/1/2023,10/1/2023,,,Developed in-house.,,,,,No,,Yes,No,Yes,,"NIH IMPAC II funded and unfunded grant application data is used. Unstructured text from project abstract, specific aims, and title are encoded and vectorized for model training and inference. Fiscal year, activity code, and RCDC terms are transformed via one-hot encoding for use in model training and inference. PII related to individuals associated with the grant is kept intact to preserve the integrity of the use case of grant application referral and the trends of researchers' focus on particular scientific areas.",Documentation is complete,,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, AI-enabled landscape analysis of New Approach Methodologies (NAMs) in biomedical research literature,Department of Health and Human Services,HHS,NIH,Health & Medical,,None of the above.,"Part of NIH Common Fund strategic planning efforts for new program. Complement Animal Research in Experimentation (Complement-ARIE) aims to identify opportunities and use trends for NAMs in biomedical research. Methods development in using genAI for literature searching and extraction applicable to other use cases. GenAI was used to survey the biomedical research literature to inform potential investment of resources and opportunities for the NIH Common Fund program. Methods are also applicable to other systematic literature review approaches.","Identify trends in use of New Approach Methodologies, gaps, and opportunities for investment by NIH.",Implementation and Assessment,Neither,1/1/2023,1/1/2023,,,Developed with contracting resources.,HHSN273201500010C; GS00Q14OADU417,,,,Yes,This use case is operated in compliance with HHS's agencywide Information Quality Act policy and procedures.,No,,No,,"Input data are publicly available, published research studies.",Documentation is widely available,,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Applying for Grants Chat Bot,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"A chatbot can significantly aid prospective grantees by providing instant, detailed information on specific grants, including eligibility criteria, application deadlines, and required documentation. It can guide users through the application process step-by-step, recommend additional resources for grant writing, and help determine eligibility to save time. Additionally, chatbots can assist in tracking application status, address common FAQs, and gather data on user needs. A chatbot streamlines the grant application process, reduces frustration, and enhances the overall experience for those seeking funding.  It can guide users through the application process step-by-step, recommend additional resources for grant writing, and help determine eligibility to save time.",Targets resources related to probing questions for end users.,Initiated,Neither,9/1/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Assessment of DTT/NTP research effectiveness,Department of Health and Human Services,HHS,NIH,Health & Medical,,None of the above.,"Dextr/LaserAI does not use PII, PHI or demographic data. Dextr is FEDRAMP certified and ATO in the cloud. Data sources for Dextr are published PDFs accessible through open source publications (e.g. PubMed Central) or through our subscriptions (e.g., NLM). Modification and customization of the DEXTR automated extraction tool has been implemented to identify and capture Division of Translational Toxicology (DTT) and National Toxicology Program (NTP) research citations in the published literature, regulations, and popular press. The DEXTR automated workflow with user-verification supports the identification of these citations that will be used to assess impact and effectiveness of DTT and NTP research. Division Directors and Branch Chiefs require an evidence-based approach to assess the effectiveness of current and past research. Such an approach would be highly valuable for research prioritization. ",Identifies how DTT/NTP research was used or impacted the publication.,Acquisition and/or Development,Neither,10/1/2024,10/1/2024,,,Developed with contracting resources.,N/A,,,,,,,,No,,"Data sources for Dextr are published PDFs accessible through open source publications (e.g. PubMed Central) or through our subscriptions (e.g., NLM).",Documentation is missing or not available,,,,,,Yes,DEXTR,Less than 6 months,No,,Yes,,No,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Assisted Referral Tool,Department of Health and Human Services,HHS,NIH,Health & Medical,,None of the above.,"Review activities of the Center for Scientific Review (CSR) are organized into Review Branches (RBs). Each RB represents a cluster of study sections around a general scientific area. Grant applications generally are assigned first to an RB, and then to a specific study section within that RB for evaluation of scientific merit. To provide assistance in assigning appropriate scientific areas for grant applications.","The Assisted Referral Tool (ART) recommends potentially appropriate study sections, based on the scientific content of a user’s grant application.  The user enters the application text and hits the Submit button to get a list of relevant study sections (scientific areas) in two groups, as “Strong” and “Possible” matches. The information provided to ART is only used to recommend study sections and is not stored or persisted.   The recommendations made by ART are solely for the benefit of the user. ",Operation and Maintenance,Neither,1/1/2014,1/1/2014,1/1/2015,,Developed in-house.,,,,,Yes,This use case is operated in compliance with HHS's agencywide Information Quality Act policy and procedures.,No,No,Yes,,IMPACII,Documentation is missing or not available,,,Yes,No – agency does not have access to source code.,,Yes,Center for Scientific Review  General Support System (CSR GSS) ,Less than 6 months,Yes,,No,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Autism Spectrum Disorder (ASD) Classification Model for Children using Deep Neural Network,Department of Health and Human Services,HHS,NIH,Health & Medical,,None of the above.,"This AI Use Case is Retired, No Longer in Use. The AI was used for evaluating whether a child around the age of 10 is exhibiting Autism Spectrum Disorder (ASD), and its severity.  This AI Use Case is Retired, No Longer in Use. The primary objective of the AI use case was to enhance the accuracy and efficiency of diagnosing Autism Spectrum Disorder (ASD) by automatically categorizing ASD using deep learning models trained on Inertial Measurement Unit (IMU) hand tracking data. The AI aimed to address the complexity and variability in ASD diagnosis, which currently relies on subjective behavioral assessments. ","The input data consists of four 3-dimensional vectors obtained from an IMU attached to the participant's hand while they perform cognitive and motor tasks. The classification result from the deep learning model outputs a value of 0 or 1, indicating ASD or typical development, respectively.",Retired,Neither,,,,8/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Automated annotation of study data using knowledge organization systems,Department of Health and Human Services,HHS,NIH,Health & Medical,,None of the above.,"Application to support automation of systematic review processes and annotation of extracted information from studies in the published literature. Automating the annotation of extracted information from environmental health studies will support joining with other structured datasets to serve as training data for predictive model development, validation of new approaches, and more robust analyses on weight of evidence of chemical effects on biological systems.",Annotation of information using knowledge organization systems.,Implementation and Assessment,Neither,1/1/2021,1/1/2022,,,Developed with contracting resources.,GS00Q14OADU417,,,,No,,No,No,No,,"Input data are publicly available, published research studies or publicly available through DTT in CEBS.",Documentation is widely available,,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,More than 12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Automated approaches for table extraction,Department of Health and Human Services,HHS,NIH,Health & Medical,,None of the above.,"The Division of Translational Toxicology (DTT) is an intramural division at NIEHS. DTT's mission is to improve public health through data and knowledge development that are translatable, predictive, and timely. DTT scientists use a variety of traditional and cutting-edge approaches to better understand how factors in our environment may impact health. Automated approaches for identifying and extracting data from tables will save time and reduce level of effort for teams evaluating published studies (epidemiology reviews or meta-analyses, hazard assessments, etc.) and ultimately impacts development of health effect conclusions.","This project developed an automated, model-based processes to reduce the time and level of effort for manual  extraction of data from tables. Published data tables are a particularly data-rich and challenging presentation of critical information in published research.",Retired,Neither,,,,1/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Automated Basic-Applied Categorization of extramural grants,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"This machine-learning algorithm is in development. Currently it uses information on NIMH-funded extramural research projects to categorize them as basic or applied research per the federal definitions for each. The data it is trained on is supplied from extramural grant applications and the same fields will be used for subsequent classification of new grants. For new grants, the algorithm will propose a categorization which can be reviewed, confirmed, or edited by an NIMH staff member. The resulting information will be used for internal, analytical purposes and annual reporting of NIMH-funded basic and applied research. The AI will NOT influence funding decisions, require or use PII, or use any other sensitive, personal, or health information. The algorithm is intended to be consistent in identifying basic and applied research, reduce burden of review by NIMH staff, and provide a complementary perspective to human review.","The algorithm produces categorization of basic and applied research that is used for internal analysis and annual reporting. The algorithm is trained on the following data from extramural grants from fiscal years (FY) 2000-2022 and will use the same for grants in FY2025 and going forward to propose a categorization of basic or applied research: Program Class Code, NIMH Division, NIH Research, Condition, and Disease Categorization (RCDC), Human Subjects Codes, Animal Subjects Code, Clinical Trial NIMH Division, text of grant title, abstracts, specific aims. These data categories are all obtained from NIH IMPACII and will be applied to 100% of NIMH extramural grants, excluding L activity codes. This constitutes approximately 1,500 new grants per FY. Algorithm performance will be assessed each year, comparing the proposed classifications to a manually reviewed sample, and input received from NIMH staff regarding edits to proposed classifications. The frequency of processing is yet to be determined. No anonymization or de-identification processes are employed as personal information is used neither to train, nor classify grants using this algorithm.",Acquisition and/or Development,Neither,6/1/2022,11/1/2022,,,Developed in-house.,,,,,,,,,No,,"The algorithm is trained on the following data from extramural grants from fiscal years (FY) 2000-2022 and will use the same for grants in FY2025 and going forward to propose a categorization of basic or applied research: Program Class Code, NIMH Division, NIH Research, Condition, and Disease Categorization (RCDC), Human Subjects Codes, Animal Subjects Code, Clinical Trial, NIMH Division, text of grant title, abstracts, and specific aims. These data are all obtained from NIH IMPACII and will be applied to 100% of NIMH extramural grants, excluding L activity codes. This constitutes approximately 1,500 new grants per FY. No anonymization or de-identification processes are employed as personal information is used neither to train nor classify grants using this algorithm.",Documentation is complete,,,,,,No,,Less than 6 months,Yes,,Yes,,No,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Automated extraction of study methods and assessment of reliability,Department of Health and Human Services,HHS,NIH,Health & Medical,,None of the above.,"Application to support automation of systematic review processes and quality assessment of studies in the published literature (environmental health studies). This project is focused on automating the identification of studies with reliable/well established methods (e.g., corresponding to regulatory guidelines) from the literature. Supervised and unsupervised methods are being applied to a) extract the methods details from known studies, and b) identify studies whose methods fulfill reliability criteria.   Identification and critical assessment of existing research is required for developing evidence-based conclusions, directing future research, and informing policy decisions by scientists, Branch Chiefs and Division Directors. Study reliability or quality are important considerations in the assessment of published research studies and are very time and labor intensive using manual approaches. ",Input: published environmental health studies; Output: identification of study methods and categorization of reliability/methods quality.,Acquisition and/or Development,Neither,1/1/2017,1/1/2018,,,Developed with contracting resources.,N/A,,,,,,,,No,,"Input data are publicly available, published research studies or publicly available through DTT in CEBS.",Documentation is complete,,,,,,No,,More than 12 months,No,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Automation of Receipt & Referral Process of NIDDK grant applications,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"This use case is centrally reported under OER's Internal Referral Module. NIDDK receives about 100 to 500 applications per week. NIDDK referral staff uses the IMPAC II QVR tool to stratify the applications between divisions based on a “Like Matrix” function to match the Division’s portfolios and the Title, Abstract, and specific aims of grant applications. This process is laborious and manual. Hence, automation was proposed. The scope of the effort is to provide a capability for NIDDK to analyze grant applications using NIDDK-specific business rules, complete the pilot, and the full implementation of the Artificial Intelligence (AI) and Natural Language Processing (NLP) tools for auto assignment of program class codes. NIDDK receives about 100 to 500 applications per week. The AI and machine learning tool will stratify the applications between programmatic divisions and Program Class Code (PCC) to match the Division’s portfolios based on the Title, Abstract, and specific aims of grant applications. It collectively saves NIDDK Program and Referral Staff 3000+ hours annually on Receipt and Referral activities. In addition, it instantly recommends PCC, allows Division designees to accept or change recommendations, tracks all grants that need PCC assignment, gives real-time information about the application status, and support business rules/generates reports.","Input: Title, Abstract, Public Health Narrative, Specific Aims of grant applications from the FY 2018 to 2022. Output: Division and Program Class Code (PCC) of grants from FY 23/24.",Operation and Maintenance,Neither,10/1/2021,10/1/2021,10/1/2023,,Developed with contracting resources.,2-DK22-560-02 and OER-23-014,,,,No,,No,No,Yes,,"Yes, It used the NIH/NIDDK grants applications data submitted by research investigators. The prediction models use the Title, Abstract, Public Health Narrative, and specific aims of grant applications from the FY 2018 to 2022 to train the models and FY 23 and F24 grants applications data were used for predicting programmatic divisions and Program Class Codes (PCC).",Documentation is complete,,,Yes,No – agency does not have access to source code.,,No,,6-12 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Best Match: New relevance search for PubMed,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"NLM is the world’s largest biomedical library. Best Match is a relevance search algorithm for NLM’s PubMed – a free search engine for biomedical literature accessed by millions of users around the world every day. Best Match increases the effectiveness of PubMed searches across the rapidly growing collection of biomedical literature to help users efficiently find the most relevant and high-quality information they need. PubMed comprises more than 37 million citations for biomedical literature from MEDLINE, life science journals, and online books.","This AI technique leverages the intelligence of our users and cutting-edge machine-learning technology as an alternative to the traditional date-sort order that appears in many traditional search engines. Trained with past user searches with dozens of relevance-ranking factors, the Best Match algorithm demonstrates state-of-the-art retrieval performance and an improved user experience. ",Operation and Maintenance,Neither,6/1/2016,6/1/2016,1/1/2023,,Developed in-house.,,,,,Yes,This use case is operated in compliance with HHS's agencywide Information Quality Act policy and procedures.,No,No,No,,"The PubMed literature collection includes articles relevant to biomedicine and the life sciences, broadly defined to encompass the information needs of those working in healthcare and life sciences. The Best Match algorithm was trained on interaction data from clicks on lists of articles returned when users search the PubMed collection with search terms.",Documentation is widely available,,,Yes,Yes – source code is publicly available.,https://github.com/ncbi-nlp/PubMed-Best-Match,No,,6-12 months,Yes,,Yes,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Biomedical Citation Selector (BmCS),Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"NLM is the world’s largest biomedical library. MEDLINE is NLM's bibliographic database that contains references to journal articles in life sciences, with a concentration on biomedicine. Through automation, NLM is able standardize article selection and reduce the amount of time it takes to process MEDLINE articles.",The output is sets of citation records that are classified as relevant to biomedicine and the life sciences. Automation of article selection allows NLM to more efficiently and effectively index and host relevant information for the public. ,Implementation and Assessment,Neither,1/1/2021,6/1/2021,,,Developed in-house.,,,,,No,,No,No,Yes,,PubMed citation data that was submitted by publishers and stored in the agency database was used.,Documentation is complete,,,Yes,Yes – source code is publicly available.,https://github.com/ncbi/biomedical-citation-selector,No,,Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Chatbot for researchers to find data sets for environmental health research efforts,Department of Health and Human Services,HHS,NIH,Health & Medical,,None of the above.,"This project is focused on the use of large language models and chatbot technology to provide a way for researchers to find data sets of relevance for environmental health research efforts. The primary objective is providing a tool that aids researchers in finding useful resources, specifically data sets, thus benefitting research efforts in finding the best data to use quickly.","The AI will use description of data sets that come from both NIH and federal data catalogs.  This will be periodically updated.  As output, the AI will provide rank order listings of recommended data sets.",Acquisition and/or Development,Neither,5/1/2024,5/1/2024,,,Developed with contracting resources.,SOAR,,,,,,,,No,,Input data are publicly available.,Documentation is widely available,,,,,,No,,Less than 6 months,No,,Yes,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Clinical Trial Predictor,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"NIGMS supports basic research that increases our understanding of biological processes and lays the foundation for advances in disease diagnosis, treatment, and prevention. NIGMS-funded scientists investigate how living systems work at a range of levels from molecules and cells to tissues and organs, in research organisms, humans, and populations. The AI tool predicts whether grant applications may involve clinical trials based on the text of their titles, abstracts, narratives, specific aims, and research strategies. ","The Clinical Trial Predictor uses an ensemble of several NLP and ML algorithms to predict whether applications may involve clinical trials based on the text of their titles, abstracts, narratives, specific aims, and research strategies. The responses are only used to make extramural officers aware of the fact that an application submitted to a NOFO that doesn't allow clinical trials may contain a clinical trial. The extramural officers then evaluate the identified applications for the presence of a clinical trial. No decisions are made by the AI, nor is the AI output used as the principle basis of a decision; it is just used to help the extramural officers identify applications containing clinical trials more quickly.",Implementation and Assessment,Neither,1/1/2023,1/1/2023,,,Developed in-house.,,,,,No,,No,No,Yes,,All data come from the internal NIH IMPAC II database.,Documentation is complete,,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, ClinicalTrials.gov Protocol Registration and Results System Review Assistant,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"ClinicalTrials.gov is a website and online database of clinical research studies and information about their results. The purpose of ClinicalTrials.gov is to provide information about clinical research studies to the public, researchers, and health care professionals. This project aims to help ClinicalTrials.gov determine whether the addition of AI could make reviewing study records more efficient and effective.",The classifier takes as input free-text related to a study’s endpoint and returns a prediction of whether one or more issues is present. ,Acquisition and/or Development,Neither,2/1/2021,11/1/2021,,,Developed with contracting resources.,GSA Contract # GS-10F-0064W NLM BPA Task Order: 75N97019A00005/75N97021F00006,,,,,,,,No,,"We utilized agency-owned data for training and evaluating the model's performance. The data consisted of clinical study record submissions from ClinicalTrials.gov's Protocol Registration and Results System (PRS) that were divided into two groups: Set A: This group included submissions that did not pass quality control review, meaning they were returned to data submitters for revision and, in certain cases, are versions of the record that were never posted on ClinicalTrials.gov (not accessible to the public). These submissions had a relevant issue identified during manual quality control review, providing examples of endpoints known to contain the specific issue. Set B: This group included submissions that were posted on ClinicalTrials.gov with endpoints that had never been labeled as having the specific issue. These submissions provided examples of endpoints that were considered not to have the specific issue.The input to the classifiers consisted of endpoint attributes, including Measure Titles, Descriptions, Time Frames, Units of Measure, and labels indicating whether the issue was present. This data did not have any personal identifiers. The data was in a structured format in the PRS, but the specific endpoint attributes were provided as free-text. We processed over 10,000 endpoints from Set A, which included examples with identified issues, and over 40,000 endpoints from Set B, which had not been flagged with issues. It is important to note that the labeling of issues in the dataset is not exhaustive. Some endpoints may present multiple issues combined and labeled as a single issue, and there is a possibility that some issues were missed by reviewers. Consequently, Set B might contain endpoints that actually have the issue despite not being labeled as such. ",Documentation is complete,,,,,,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Collections Summarization Chatbot,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"The artificial intelligence (AI) use case involves developing a chatbot that can answer contextual questions about collection objects that users are browsing within the NLM Digital Collections. The chatbot will not handle inquiries unrelated to collection objects or queries outside its defined scope. It utilizes the Retrieval-Augmented Generation (RAG) framework on Amazon Web Services (AWS) to generate summaries or detailed answers based on the content users are viewing. The chatbot interface will integrate with existing NLM web technologies, activating when a user spends an extended time on a resource page. The AI system will use text data from the collection objects and associated metadata to generate responses. The primary business area for deployment will be within NLM’s digital services, focusing on enhancing user interaction and accessibility.  The main goal is to provide users with concise summaries and detailed answers about NLM Digital Collections.","The input consists of text from documents or books within the Collection objects that users are viewing. The frequency of AI processing depends on how long users spend on the resource page. If users linger, the AI chatbot will prompt them with a summary of the resource they are viewing. The AI-generated output will be a summary of the resource presented in text format. ",Initiated,Neither,7/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Computed Author: author name disambiguation for PubMed,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"NLM is the world’s largest biomedical library. PubMed comprises more than 37 million citations for biomedical literature from MEDLINE, life science journals, and online books. NLM developed a machine-learning method to score the features for disambiguating a pair of papers with ambiguous names. Subsequently, agglomerative clustering is employed to collect all papers belonging to the same authors from those classified pairs. Disambiguation performance is evaluated with manual verification of random samples of pairs from clustering results, with a higher accuracy than other state-of-the-art methods. It has been integrated into PubMed to facilitate author name searches.","PubMed users frequently use author names in queries for retrieving scientific literature. However, author name ambiguity (different authors share the same name) may lead to irrelevant retrieval results.",Operation and Maintenance,Neither,5/1/2009,5/1/2009,1/1/2023,,Developed in-house.,,,,,No,,No,No,No,,Searches of the PubMed literature collection using author names were used to create training and testing data sets consisting of article sets.,Documentation is widely available,,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, COVID-19 Pandemic Vulnerability Index Dashboard,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"The dashboard creates risk profiles, called PVI scorecards, for every county in the United States, continuously updated with the latest data that summarize and visualize overall disease risk.  Aims to reduce the impact of COVID-19 on vulnerable populations by aiding equitable decision-making on resource allocation and implementation of local- and state-level interventions. Available at https://covid19pvi.niehs.nih.gov/. ","Inputs are location-specific current infection rates, baseline population concentration, current interventions, and health and environmental vulnerabilities. Outputs are data visualizations and vulnerability estimates.",Operation and Maintenance,Neither,3/1/2020,3/1/2020,3/1/2020,,Developed in-house.,,,,,Yes,This use case is operated in compliance with HHS's agencywide Information Quality Act policy and procedures.,No,No,No,,Source data is derived from publicly available resources detailed here: https://www.niehs.nih.gov/research/programs/coronavirus/covid19pvi/details/.,Documentation is missing or not available,,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,No,,No,,No,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, CSR Public Chatbot (CPC),Department of Health and Human Services,HHS,NIH,Health & Medical,,None of the above.,"This tool is a chatbot that answers questions from potential applicants and reviewers. Instead of providing lists of links to look through, the chatbot provides answers with links where the answers were drawn from. Source materials are limited to official government web pages, and a disclaimer is included cautioning the user to check the source materials to confirm the accuracy of the answers. This chatbot is intended to provide answers to the public about questions related to NIH peer review policies and procedures. ","Input: Applicant/Reviewer Questions. Output: Answers to the questions ",Operation and Maintenance,Neither,1/1/2021,1/1/2021,1/1/2021,,Developed in-house.,,,,,Yes,This use case is operated in compliance with HHS's agencywide Information Quality Act policy and procedures.,No,No,Yes,,Applicant FAQs and publicly available content from public.csr.nih.gov website.,Documentation is missing or not available,,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Center for Scientific Review  General Support System (CSR GSS),Less than 6 months,Yes,,No,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, DAIT AIDS-Related Research Solution,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"NIAID administers HIV/AIDS grant portfolios. The incoming grant applications are ranked based on these predictions and more highly-ranked applications are prioritized for review.","The tool uses natural language processing text extraction and classification algorithms to predict both priority (High, Medium, Low) and category (Area of research) for a grant application. After DAIT awards grants, they review all awarded grants to determine if any of the awardees are conducting research that could be considered AIDS-Related. NIH’s Office of AIDS Research (OAR) will fund all or part of the grants if the research is considered AIDS-Related. The DAIT AIDS-Related Research solution (DAIT ARR), categorizes the awarded grants into research topics. This categorization is based on the title and abstract of the grant application.",Operation and Maintenance,Neither,1/1/2017,1/1/2017,1/1/2018,,Developed with contracting resources.,Not Reported,,,,No,,No,No,Yes,,"Yes, a dataset was curated to train the model and is evaluated manually by user input.",Documentation is complete,,,Yes,Yes – source code is publicly available.,http://tfs.niaid.nih.gov:8080/tfs/BI/DAIT_ARR/_git/grants?path=/main.py,Yes,Not Reported,Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Detecting Overlapping Science (DOS),Department of Health and Human Services,HHS,NIH,Health & Medical,,None of the above.,"This tool detects applications that represent potential duplicate funding (funding the same research in different projects). It examines applications as they are submitted to the NIH and sends a report to relevant personnel in the agency. Detect and prevent duplicate funding with real-time examination of incoming grant applications.","Input: Text of grant applications. Output: Pairs of possible duplication.",Operation and Maintenance,Neither,1/1/2023,1/1/2023,1/1/2023,,Developed in-house.,,,,,No,,No,No,Yes,,IMPACII,Documentation is missing or not available,,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Center for Scientific Review General Support System (CSR GSS),Less than 6 months,Yes,,No,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Detection of Implementation Science focus within incoming grant applications,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"Grants management oversight and administration. Assign the grant application to a particular division for routine grants management oversight and administration.","This tool uses natural language processing and machine learning to calculate an Implementation Science score that is used to predict if a newly submitted grant application proposes to use science that can be categorized as ""Implementation Science.""",Operation and Maintenance,Neither,1/1/2019,1/1/2019,1/1/2020,,Developed with contracting resources.,N/A,,,,No,,No,No,Yes,,Leverages NIH application data from IRDB.,Documentation is complete,,,No,No – agency does not have access to source code.,,Yes,Dimensions for NIH and Analyst Support Infrastructure (ATO Letter - https://nih.sharepoint.com/:b:/s/NHLBI-ITAC-SECURITY/Eb_TarkpZPRCnX5RsGEBgQ8BgK4XC2_hjlsAVeHHLEjdww),Less than 6 months,Yes,,No,,No,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, DEXTR - automated data extraction tool,Department of Health and Human Services,HHS,NIH,Health & Medical,,None of the above.,"Dextr/LaserAI does not use PII, PHI or demographic data. Dextr is FEDRAMP certified and ATO in the cloud. Data sources for Dextr are published PDFs accessible through open source publications (e.g. PubMed Central) or through our subscriptions (e.g., NLM). The primary objective of the DEXTR/Laser AI use case is to streamline the process of data extraction from published studies for literature reviews, addressing the key problem of time and labor intensity involved in manual data identification and capture. DEXTR leverages natural language processing (NLP) models, regular expressions and large language models to automatically identify and predict pertinent entities of data extraction within scientific studies. By doing so, it reduces the burden of manual extraction while still supporting user verification, offering a semi-automated approach that ensures accuracy and quality control. One of the key tasks DEXTR enhances is the annotation of studies during the normal workflow, allowing the tool to generate annotated corpora for the development of future models. Dextr also provides automated extraction of information from tables in PDFs expanding the supported data extraction capabilities.  As a dynamic platform for multiple AI projects, DEXTR/LaserAI is continually being optimized and expanded to support a growing number of use cases. This automation not only accelerates data extraction but also aids in creating a robust, scalable system for handling the ever-increasing volume of published research. The anticipated positive outcomes include improved efficiency and accuracy in data extraction, which is critical for developing evidence-based conclusions, guiding future research directions, and informing policy decisions. By enabling faster and more comprehensive reviews of existing literature the tool has the potential to significantly enhance the ability to process large volumes of scientific data.","DEXTR leverages natural language processing (NLP) models, regular expressions and large language models to automatically identify and predict pertinent entities of data extraction within scientific studies.",Implementation and Assessment,Neither,8/1/2020,8/1/2020,,,Developed with contracting resources.,N/A,,,,No,,No,No,No,,"Data sources for Dextr are published PDFs that are accessible through open source publications (e.g. PubMed Central) or through our subscriptions (e.g., NLM).",Documentation is missing or not available,,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Dextr is FEDRAMP certified and ATO in the cloud,Less than 6 months,No,,No,,No,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Division of International Services Customer Service Chatbot,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"The initial version is designed to allow both foreign nationals and administrative officers (AOs) to interactively receive accurate answers to general questions about immigration policy and operations managed by NIH Division of International Services. The AI data store is compiled from DIS Frequently Asked questions and other relevant policy and general information. The benefit of the AI is to alleviate manual calls to the division for questions that can be covered more efficiently and accurately.","Generates text, images and data. Input data source is a local database. Frequency is TBD.",Initiated,Neither,6/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Enhance NLM Website Accessibility,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"NLM is pilot testing the use of artificial intelligence (AI) to enhance the accessibility of the NLM website in alignment with Section 508 standards.  This initiative is critical to mitigate the enterprise risk associated with non-compliance with Section 508 standards covering web sites. The AI will focus solely on identifying and resolving accessibility errors associated with Section 508 standards for websites, including suggestions from the U.S. Web Design System framework when possible. The AI is currently being developed behind NIH's firewall for a select internal user base. The tool will not make direct changes to general web design or web content. The AI will integrate with existing web content management systems and accessibility compliance tools (like pa11y).  The AI will use website data (HTML, CSS, JavaScript) and accessibility compliance standards as input and will be deployed in the web development and IT departments, particularly those focused on accessibility and compliance. The project may reduce costs and time associated with manual accessibility evaluation and remediation. Additionally, it is expected to enhance the accuracy and quality of accessibility features on the website. Importantly, the project aims to improve the user experience by making the site more accessible to a broader audience, thus increasing user access and discovery. Ultimately, this initiative will help reduce the enterprise risk associated with non-compliance and ensure that NLM's website meets the evolving standards for government web accessibility.","Inputs: Website data (HTML, CSS, JavaScript), accessibility standards. Outputs: Accessibility issue reports, suggested code fixes, task lists for implementation. Frequency: The AI performs both site-wide scans and single URL audits on demand. ",Acquisition and/or Development,Neither,11/1/2023,5/1/2023,,,Developed with contracting resources.,75N97023A00004/75N97023F00005,,,,,,,,Yes,,"Website data (HTML, CSS, JavaScript), accessibility standards. Measures taken to ensure data accuracy and reliability: grounding, testing, evaluation, NIST framework.",Documentation is complete,,,,,,No,,Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Enhancing Responses to Customer Questions about NLM Products,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"NLM is piloting the creation of an application which uses the GPT family of models from OpenAI (via Azure) and data retrieved from public facing NIH/NLM web domains to answer questions about select NLM products. This is currently being developed behind NIH's firewall for a select internal user base. The artificial intelligence (AI) will generate responses to questions but will not engage in direct conversations with customers. Additional safeguards would make sense in a production environment. The AI will integrate with a knowledge base and use text data from user technical questions and knowledge base articles. It will be deployed within customer service and technical support offices. Primary Objectives: Provide accurate and timely responses to technical questions about NLM products. Anticipated Positive Outcomes: Improved efficiency and accuracy in customer service, better user experience, and higher satisfaction rates.","Inputs: User technical questions, knowledge base articles. Outputs: AI-generated responses, gold standard responses, knowledge base links. Frequency: Responses generated in real-time based on user inquiries.",Acquisition and/or Development,Neither,11/1/2023,5/1/2023,,,Developed with contracting resources.,75N97023A00004/75N97023F00008,,,,,,,,Yes,,"Website data (HTML, CSS, JavaScript), accessibility standards. Measures taken to ensure data accuracy and reliability: grounding, testing, evaluation, NIST framework.",Documentation is complete,,,,,,No,,Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Environmental health research annotation for model development,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"By creating annotated datasets that account for diverse study types, the project will enable the development of AI and machine-learning models that can significantly reduce the time and effort required to identify and capture critical data from environmental health studies. This improvement will streamline tasks for teams conducting literature assessments and reviews, improving their efficiency. The primary objective of this AI use case is to enhance the development of datasets specifically focused on toxicology and environmental health. The key problem it aims to address is the current scarcity of relevant training datasets or corpora, which limits the ability to create effective AI models for identifying and extracting environmental health data from published studies. This challenge is compounded by the diverse nature of environmental health research, which includes various study designs such as epidemiological studies, in vitro research, exposure assessments, and experimental animal studies.  ","Input: published environmental health studies. Output: annotated corpus to support model development (e.g., BRAT format).",Acquisition and/or Development,Neither,1/1/2020,1/1/2020,,,Developed with contracting resources.,N/A,,,,,,,,No,,N/A,Documentation is missing or not available,,,,,,No,,Less than 6 months,Yes,,No,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Expansion of Generative AI (GenAI) Caption Generation for all Collections Videos,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"The artificial intelligence (AI) use case builds on an existing Generative AI (GenAI) workflow in NLM Digital Collections that generates transcripts and captions for all videos stored in the Digital Repository dark archive. The expansion of this workflow will apply to all videos and films targeted for Digital Collections. The AI will not be used for content outside of these collections or for non-video materials, as well as already captioned videos. It will integrate with existing video processing systems to automate the captioning process. The types of data utilized will include video files and associated metadata. The deployment will occur within NLM’s Digital Collections management area, with the primary goals being cost savings by reducing reliance on third-party vendors and standardizing video caption workflows across all collections. The objective is to expand AI-driven captioning across all videos in the NLM Digital Collections, ensuring standardized and accuracy. ",The inputs consist of video or audio mp4 or mp3 format comes from Collections videos that don't have captions. The AI generated captions will be in text format.,Implementation and Assessment,Neither,2/1/2020,3/1/2020,,,Developed with contracting resources.,HHSN316201200013W/75N97019F00066,,,,Yes,This use case is operated in compliance with HHS's agencywide Information Quality Act policy and procedures.,No,No,Yes,,"The agency-owned data used in this AI use case consists of U-Matic videos in MP4 format, hosted on collections.nlm.nih.gov. Each month, 8 uncaptioned U-Matic videos of various length are selected for processing, where the AI-generated captions are applied. The audio is extracted from the video files and used as input for AWS Transcribe, which is a pre-trained model that generates captions. During the QA process, there is a human reviews process that checks the quality and accuracy of the output captions, and a disclaimer is added to the videos indicating that the captions were AI-generated.",Documentation is complete,,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,No,,Yes,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Federal IT Acquisition Reform Act (FITARA) Tool,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"NIAID administers contracts to execute mission initiatives. The tool's classifications inform SOW clarifications/revisions relevant to the FITARA.",The tool classifies contract statement of work (SOW) documents as being IT-related or not. ,Operation and Maintenance,Neither,1/1/2017,1/1/2017,7/1/2017,,Developed with contracting resources.,Not Reported,,,,No,,No,No,Yes,,"Yes, a dataset was curated using NIAID SOWs which were manually labelled to train the classification model.",Documentation is complete,,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Not Reported,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Generative AI (GenAI) Still Image Tagging,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"The artificial intelligence (AI) use case will focus on identifying still images within the NLM Digital Collections that lack descriptions. This use case will employ Generative AI (GenAI) tools such as Amazon Web Services (AWS) Rekognition and/or AWS Bedrock to identify concepts and embedded text within these images. The AI system will not generate metadata for images that already have detailed descriptions or analyze images outside the NLM Digital Collections. The AI will connect with existing NLM metadata management systems, where the generated metadata will be stored and indexed. The primary types of data used will include image files from the digital collections and textual data derived from embedded text within those images. The AI system will be deployed within the NLM's business area, focusing specifically on enhancing the search and retrieval capabilities of the NLM Digital Collections. The use case aims to automatically generate accurate metadata for under-described images in the NLM Digital Collections, enhancing their searchability and discoverability. This improvement will increase access to valuable medical and scientific resources, supporting research and public health initiatives. ","Input consists of image binaries (tif, jpeg or PDF format) served from Collections website can be hosted on S3; Amount consists of untagged images within the repository, ballparked about 75,000 image objects, then about 10-100 per month based on monthly content ingestions. Output should be AI generated classifications tags and/or AI generated image summary in text form. ",Initiated,Neither,5/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Grant Application Subject-Matter Classification Tool,Department of Health and Human Services,HHS,NIH,Health & Medical,,None of the above.,"The algorithm is used to recommend a specific scientific area within the Division's portfolio with a specific confidence level. This reduces the time it takes to assign a Program Class Code and personnel assignment to every grant application received by NIEHS. It searches for key words within the title, abstract and specific aims in each grant application to determine the best scientific category. It is not used in combination with any other data sets. It uses grant application data from IMPAC II, which is imported into the Grants Funding Decision Tool (a NIEHS Custom Product). Natural language processing of grant applications to assign scientific topics and corresponding Program Officer staff.  The subject-matter classification tool improves the efficiency of the grant administration process by automatically assigning incoming applications a subject matter classification based on the text content of their abstracts using a model derived from past human-classified examples. This classification helps route each application to the program officer with the greatest expertise in the inferred field of study.",Training set consists of an Excel spreadsheet containing human-classified grant proposal abstracts. Input file is an Excel table containing new unclassified grant proposal abstracts. Output is input file updated to include tool-assigned subject matter classifications.,Operation and Maintenance,Neither,1/1/2016,1/1/2016,2/1/2020,,Developed with contracting resources.,Not Reported,,,,No,,No,No,No,,"Input data is provided to NIEHS by grant applicants, via the IMPAC II database.",Documentation is complete,,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,No,,No,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, HIV-related grant classifier tool,Department of Health and Human Services,HHS,NIH,Health & Medical,,None of the above.,"A front-end application for scientific staff to input grant information, which then runs an automated algorithm to classify HIV-related grants.  The tool allows OAR staff to determine the best categorization for a grant application after reviewing data and visualization of confidence levels through a heat map. Natural language processing of grant applications to assign scientific topics and corresponding Program Officer staff.  The subject-matter classification tool improves the efficiency of the grant administration process by automatically assigning incoming applications a subject matter classification based on the text content of their abstracts using a model derived from past human-classified examples. This classification helps route each application to the program officer with the greatest expertise in the inferred field of study.","Input: Grant text data. Output: Predicted categorization.",Implementation and Assessment,Neither,11/1/2022,12/1/2022,,,Developed in-house.,,,,,No,,No,No,No,,"Grant text data (Title, Abstract and Specific Aims).",Documentation is complete,,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,No,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Improving Customer Response,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"NLM is pilot testing the utilization of artificial intelligence (AI) to assist Customer Service Representatives (CSRs) in drafting responses to customer inquiries. It is currently being developed behind NIH's firewall for a select internal user base. The AI is focused on generating draft responses for customer service but will not make final decisions or engage in direct communication with customers. CSRs are instructed to not enter personally identified info into the AI system. In the event this info is entered, the AI system is automatically prompted to not include any personally identified information in its generated response. Additional safeguards would make sense in a production environment or if it was released to the public. The AI will integrate with customer service platforms and databases like Medline Plus and DailyMed. The AI will use text data from customer inquiries and publicly accessible NLM-managed biomedical information services and be deployed within the customer service department.  The project aims to enhance customer response metrics, increase customer satisfaction, and improve the efficiency of handling customer service inquiries.","Inputs: Customer inquiries, publicly accessible, NLM-managed biomedical information services (such as Medline Plus, DailyMed, etc.). Outputs: Draft responses, resource links, disclaimers. Frequency: Responses generated in real-time as inquiries are received.",Acquisition and/or Development,Neither,11/1/2023,5/1/2023,,,Developed with contracting resources.,75N97023A00004/75N97023F00006,,,,,,,,Yes,,"Website data (HTML, CSS, JavaScript), accessibility standards. Measures taken to ensure data accuracy and reliability: grounding, testing, evaluation, NIST framework.",Documentation is complete,,,,,,No,,Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Improving Metadata Retrieval and Transformation for Metadata Management,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"The Dataset Catalog (catalog) is a catalog of biomedical datasets from various repositories for users to search, discover, retrieve, and connect with datasets to accelerate scientific research. Metadata from included repositories and their datasets are needed to appropriately index them in the catalog. This manual process can be labor intense, and thus NLM is pilot testing the development of a working interface for internal users to retrieve the metadata information from external repositories' websites and then transform the data to a specific format used in internal metadata management for the Dataset Catalog. This is currently being developed behind NIH's firewall for a select internal user base. The artificial intelligence (AI) will automate the retrieval and transformation of metadata but will not replace human oversight in critical metadata curation tasks. Additional safeguards would make sense in a production environment. The AI uses information from publicly available biomedical data repository websites and internal metadata management systems. The AI uses metadata from biomedical repositories and will be deployed within data management and IT departments, particularly those involved in cataloging and metadata management. Primary Objectives: Streamline metadata retrieval and transformation to improve efficiency and accuracy. Anticipated Positive Outcomes: Resource savings, enhanced data quality, and improved management of the Data Set Catalog.","Inputs: Metadata from various biomedical repositories, internal schemas. Outputs: Transformed metadata, Python scripts, user-friendly tools for metadata retrieval. Frequency: Continuous retrieval and transformation as new metadata is ingested.",Acquisition and/or Development,Neither,11/1/2023,5/1/2023,,,Developed with contracting resources.,75N97023A00004/75N97023F00007,,,,,,,,Yes,,"Website data (HTML, CSS, JavaScript), accessibility standards. Measures taken to ensure data accuracy and reliability: grounding, testing, evaluation, NIST framework.",Documentation is complete,,,,,,No,,Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Internal Referral Module ,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"The NIH Extramural Research Program uses the eRA system to efficiently award and manage grants. The AI system is designed for use by IC ‘internal referral’ officers who manage a pool of applications and are responsible for assigning the applications to program officials. It creates efficiencies by automating the ‘internal referral’ assignments, currently done manually on spreadsheets by the ICs. The inputs are grant applications, the outputs are referrals to Program Officers, Program Class Codes, Organizational units - Divisions and Branches and Scientific Research Clusters.","The Internal Referral initiative automates a manual process by using AI and NLP capabilities to help assign grant applications to NIH Institute and Center Program Officers to make informed decisions. The inputs are grant applications, the outputs are referrals to Program Officers, Program Class Codes, Organizational units - Divisions and Branches and Scientific Research Clusters.",Operation and Maintenance,Neither,1/1/2018,1/1/2019,2/1/2023,,Developed with contracting resources.,"N/A, The work is performed across multiple eRA contracts.",,,,No,,No,No,Yes,,"We used eRA grant application data for all fine tuning and optimization for the models. Specially, we extract the title, abstract, specific aims and public health narrative to train our models for prediction. ",Documentation is complete,,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,eRA,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, IT System Outage Alerts,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"Use Generative AI to create maintenance and IT system outage alerts that will be sent to NIDCD staff. General communication to the NIDCD staff. The GenAI tool will reduce staff time to create and distribute e-mail alerts.","Communicates alerts for IT System outages. Output: System, building, day, time.",Initiated,Neither,10/1/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, JIT Automated Calculator (JAC),Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"NIGMS supports basic research that increases our understanding of biological processes and lays the foundation for advances in disease diagnosis, treatment, and prevention. NIGMS-funded scientists investigate how living systems work at a range of levels from molecules and cells to tissues and organs, in research organisms, humans, and populations. NIGMS needs to know how much total support an investigator has to ensure that it is not funding Principal Investigators who are already adequately resourced. The tool can determine how much funding a Principal Investigators has been awarded.",The JIT Automated Calculator (JAC) uses NLP to parse Just-In-Time (JIT) Other Support forms and determine how much outside support PIs are receiving from sources other than the pending application.,Operation and Maintenance,Neither,3/1/2024,3/1/2023,5/1/2023,,Developed in-house.,,,,,No,,No,No,Yes,,All data come from the internal NIH IMPAC II database.,Documentation is missing or not available,,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Large Language Automation,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"Automate use cases as indicated below. 1. Automate BOT to locate and extract Resource Sharing Plan from post awarded grant application and feed into Microsoft Azure AI to teach it to determine if grant application as written has met compliance. The RFA provides guidelines for grantees. 2. Generate summary for published papers. 3. Generate summary for papers for concept development, research data analysis or other internal work. 4. Generate summary as part of a preliminary review of documents. 5. Generate summary of public meetings. 6. Utilize existing MS Excel worksheets to automate templated emails. 7. Generate FAQs. 8. Produce training materials.  The RFA:  1) provides guidelines for grantees, 2) generates summary for published papers, 3) generates summary for papers for concept development, research data analysis or other internal work, 4) generates summary as part of a preliminary review of documents, 5) generates summary of public meetings, 6) utilizes existing MS Excel worksheets to automate templated emails, 7) generates FAQs, and 8) produces training materials. ","Input: Push data into Microsoft Azure AI from QVR post awarded grant application data. Output: Integration of Microsoft Azure AI with UiPath. ",Initiated,Neither,10/1/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Leveraging User-Generated Content for Digital Behavioral Interventions,Department of Health and Human Services,HHS,NIH,Health & Medical,,None of the above.,"The project uses AI-based automated text classification technology to build a predictive model to categorize smoking-related tweets into six distinct categories (e.g., quitting strategies). Specifically, the project uses the AI-based BERT pre-trained language model, which will be further fine-tuned using neural network models to capture the contextual nuances of our use case. AI will also be used to select data for the training dataset that will form the basis of the predictive model. Then, we use ChatGPT to classify extracted and labelled tweets according to its sentiment, presence of sarcasm, presence of irony, presence of emojis.  The primary objective of the project is to classify thousands of tweets to six categories (e.g., strategies for quitting, quitting benefits) and according to their sentiment and presence of irony, sarcasm, and emojis. Tweets that are positive, not ironic, nor sarcastic would then be tested with target audiences for message effectiveness and possible integration in a smoking cessation intervention.","Input: Publicly available tweets. Output: sentiment (very positive, positive, neutral, negative, very negative), presence of sarcasm (yes, no), presence of irony (yes, no), presence of emojis (yes, no - and if yes, a description of the emojis present in the tweet).",Implementation and Assessment,Neither,1/1/2024,1/1/2024,,,Developed with contracting resources.,N/A,,,,No,,No,No,No,,Corpus of 5 million tweets extracted for the purposes of the research by a colleague who had access to twitter archives.,Documentation is missing or not available,,,No,No – agency does not have access to source code.,,No,,More than 12 months,No,,No,,No,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, LLM Support for Admin Services,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"This project aims to explore the integration of large language models (LLMs) into administrative operations, aiming to enhance efficiency, accuracy, and productivity. By leveraging LLMs, repetitive tasks can be automated including data analysis, documentation drafting, and customer support, thereby freeing up valuable time for administrative personnel to focus on more strategic initiatives. Additionally, LLMs can facilitate natural language processing (NLP) for documentation, ensuring consistency and accuracy in communication across various channels. Through process automation, policy monitoring, and virtual assistance, LLMs offer the potential to streamline our administrative workflow, reduce manual workload, and mitigate the risk of errors. This assessment will provide valuable insights into the feasibility and effectiveness of integrating LLMs into our administrative framework, ultimately optimizing our operations and enhancing organizational performance. It will reduce the burden of administrative tasks, faster response to administrative inquiries, fewer resources required to support administrative operations.","Uses NLM administrative and operational data to output procedures, forms, schedules, etc.",Initiated,Neither,5/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Mapping Sequence Data to Research Outcomes,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"A proof-of-principle analysis is being conducted to identify downstream research products related to sequence records included in NLM-managed repositories. To identify sequence records in NLM-NCBI-managed data repositories and related research products to inform operations and prioritize resources.",Sequence records included in NLM-NCBI-managed data repositories and abstracts for citations records indexed in PubMed; outputs are text summaries identifying in a yes/no fashion if research products were identified,Initiated,Neither,12/1/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Medical Text Indexer-NeXt Generation (MTIX) MEDLINE Indexing,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"Machine learning-based system for the automated indexing of MEDLINE citation records using Medical Subject Heading (MeSH) terms. Automated indexing is achieved using a multi-stage neural text ranking approach. MeSH indexing has been shown to improve information retrieval in PubMed, but manual indexing was not sustainable due to the increasing volume of published biomedical literature. Automated indexing allows for cost-effective and timely indexing of MEDLINE citation records.","The input is PubMed citation data including the article title and abstract, and the output is a set of MeSH terms describing the topic of the article.",Operation and Maintenance,Neither,1/1/2021,1/1/2022,5/1/2023,,Developed with contracting resources.,Black Canyon Consulting LLC 47QTCA18D00JA/75N97023F00098,,,,No,,No,No,Yes,,"The MTIX dataset is approximately 10 million PubMed MEDLINE citations published after 2006. It is publicly available data, used for training and evaluation of the MeSH terms predicted by the algorithm. ",Documentation is complete,,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, MetaMap,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"NLM is the world’s largest biomedical library. MetaMap is a program providing access from biomedical text to the concepts in the Unified Medical Language System Metathesaurus. MetaMap uses NLP to provide a link between the text of biomedical literature and the knowledge.","Metamap is used to map biomedical text to the Unified Medical Language System Metathesaurus or, equivalently, to discover Metathesaurus concepts referred to in text. ",Operation and Maintenance,Neither,1/1/1996,1/1/1996,1/1/1998,,Developed with contracting resources.,GSA Contract Number: 47QTCA18D00JA,,,,No,,No,No,Yes,,"UMLS Metathesaurus is used by MetaMap to map the incoming text to concepts that exist in the UMLS Metathesaurus.  MEDLINE citations from the MEDLINE baseline were used in the development, training, and testing of MetaMap.   ",Documentation is missing or not available,,,Yes,Yes – source code is publicly available.,https://lhncbc.nlm.nih.gov/ii/tools/MetaMap/run-locally/Sources.html,No,,Less than 6 months,Yes,,No,,Yes,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, MicroStrategy Evaluation,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"The software to be evaluated should provide a business intelligence/reporting/visualization interface for possible use for 2 systems (Azure DevOps and OneStream). This software should provide robust development capabilities for manually-generated reports and AI capabilities for end-users to generate additional reports without the need for development.  This software should provide robust development capabilities for manually-generated reports and AI capabilities for end-users to generate additional reports without the need for development. ","Input: Financial data from OneStream and Project Management data from Azure DevOps. Output: Data driven business intelligence dashboards. ",Initiated,Neither,10/1/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, MTIX,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"NLM is the world’s largest biomedical library. MEDLINE is NLM's bibliographic database that contains references to journal articles in life sciences, with a concentration on biomedicine. This is a machine learning-based system for the automated indexing of MEDLINE articles with Medical Subject Headings (MeSH) terms. Automated indexing is achieved using a multi-stage neural text ranking approach. ",Automated indexing allows for cost-effective and timely indexing of MEDLINE articles.,Implementation and Assessment,Neither,1/1/2021,1/1/2022,,,Developed with contracting resources.,Black Canyon Consulting LLC 47QTCA18D00JA/75N97023F00098,,,,No,,No,No,Yes,,"The MTIX dataset is approximately 10 million PubMed MEDLINE citations published after 2006. It is publicly available data, used for training and evaluation of the MeSH terms predicted by the algorithm. ",Documentation is complete,,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, NanCI: Connecting Scientists,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"NCI is helping to connect scientists. NanCI is the app for biomedical scientists that helps discover scientific papers, build a career network, and explore events. ",NanCI is a new mobile application that uses machine learning algorithms to match users’ interests and provide a unique experience by recommending tailored content.,Operation and Maintenance,Neither,1/1/2021,1/1/2022,1/1/2024,,Developed with contracting resources.,Google & Barnacle.ai via STRIDES mechanism,,,,No,,No,,No,,N/A,Documentation is missing or not available,,,No,No – agency does not have access to source code.,,No,,6-12 months,No,,No,,No,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, NBS Virtual Assistant,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"The NIH Business System (NBS) is NIH's financial system. Streamlined Service Desk Operations. The AI-powered chatbot can handle routine user inquiries, reducing the burden on IT service desk. Provides self-service options for users to resolve issues independently.","The NIH Business System (NBS) Assistant is an AI-powered chatbot to redefine how NBS users engage, and to offer seamless support and personalized assistance across various NBS workstreams. NBS is NIH's financial system.",Implementation and Assessment,Neither,1/1/2024,7/1/2024,,,Developed with contracting resources.,Software - 75N97024F00051. System Integrator - 75N97022F00084,,,,No,,No,No,No,,"G-Invoicing: This dataset includes detailed information related to the G-Invoicing process, such as transaction records and invoicing workflows documentation. User Provisioning: This includes data related to user access management, including roles, permissions, and user activity logs within the NIH systems. PRISM-Related Job Aids and Documents: This dataset comprises job aids, guides, and documentation related to PRISM, which supports various administrative and operational functions at NIH.",Documentation is complete,,,No,"Yes – agency has access to source code, but it is not public.",,Yes,NBS AI will be part of the NBS Cloud (NBSC) ATO. System name is NBSC.,6-12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, "NCI-DOE Collaboration, MOSSAIC project (Modeling Outcomes using Surveillance Data and Scalable AI for Cancer)",Department of Health and Human Services,HHS,NIH,Health & Medical,,None of the above.,"MOSSAIC applies deep learning natural language processing (NLP) and foundation models to population-based cancer data collected by NCI's Surveillance, Epidemiology, and End Results (SEER) program. DOE's Oak Ridge National Lab (ORNL) has data use agreements (DUAs) with multiple SEER registries to access and train models using SEER data. To date, the MOSSAIC pathology-coding API is deployed in 22 SEER registries, moving the US towards near real-time cancer incidence reporting. Other APIs are still in the validation phase or research and development phase. MOSSAIC enhances the infrastructure of the SEER cancer registries by providing tools that can increase the efficiency and accuracy of manual data abstraction by automatically extracting cancer surveillance data elements.  SEER registries receive millions of unstructured clinical text documents that must be manually reviewed, leading to a lag in reporting of US cancer incidence trends.  Automated tools such as those developed by MOSSAIC will help us achieve near real-time incidence trends and ultimately a more meaningful report card on the status of cancer in the US.","Input: unstructured (free text) cancer pathology reports. Output: varies depending on the algorithm but generally a predicted class (eg, tumor site) and associated relative confidence score that can be used to tune accuracy.",Implementation and Assessment,Neither,1/1/2017,10/1/2017,,,Developed with contracting resources.,Developed under an Interagency Agreement with the US Department of Energy,,,,No,,Yes,No,No,,"Data is owned by the NCI SEER registries, which are funded by the NCI",Documentation is complete,,,No,Yes – source code is publicly available.,https://computational.cancer.gov/view-model-new?f%5B0%5D=project%3Amossaic&search_api_fulltext=&sort_by=title_1&sort_order=ASC&items_per_page=10,No,,Less than 6 months,No,,No,,No,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, NHLBI Chat,Department of Health and Human Services,HHS,NIH,Health & Medical,,None of the above.,"NHLBI Chat is a secure LLM tooling providing access to the Azure OpenAI API so that all NHLBI staff can explore generative AI for their day-to-day need. NHLBI Chat is a secure LLM tooling providing access to the Azure OpenAI API so that all NHLBI staff can explore generative AI for their day-to-day need.",The Azure OpenAI API accepts text as input and return text as output. Users enter text through a chat interface in a website.,Operation and Maintenance,Neither,5/1/2023,5/1/2023,9/1/2024,,Developed in-house.,,,,,No,,No,No,No,,N/A,Documentation is missing or not available,,,Yes,Yes – source code is publicly available.,https://github.com/NHLBI/LLM_Chat_Interface,Yes,NHLBI Chat - https://trac.nhlbi.nih.gov/trac/openTracRequest?tracIDNumber=47124&type=simple ,Less than 6 months,Yes,,No,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, NIAID GenAI Toolkit,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"The NIAID GenAI Chat and Document Bot are Azure-hosted GenAI applications that leverage commercially available large language models from the Azure OpenAI Service to assist NIAID employees in a variety of task such as content generation, summarization, etc. The Azure-hosted NIAID GenAI Chat and Document Bot helps employees be more efficient with a wide variety of administrative tasks, such as summarizing/querying documents, drafting emails, and creating presentation outlines​, etc. ","Input: natural text in the form of user questions, user uploaded documents.  Output: Generated text in the form of answers to user questions, generated answers (summaries/queries) based on user documents. ",Operation and Maintenance,Neither,2/1/2024,2/1/2024,7/1/2024,,Developed with contracting resources.,N/A,,,,No,,No,No,Yes,,"No, training, fine-tuning, and performance evaluation was not conducted.",Documentation is missing or not available,,,Yes,Yes – source code is publicly available.,https://github.com/SEMOSS/Semoss,Yes,Not Reported,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, NIAMS AI Chatbot Pilot,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"To develop a secured AI Chatbot environment where NIAMS staff can 1) conduct general research, 2) help staff in writing grants/publications/papers (e.g. foreign researcher with their English grammar, sentence structure, etc.), 3) scientific and IT code generation, 4) summarizing meeting minutes and 5) providing a platform to learn how AI can be used in their daily work tasks. Aside from what is described in 'Use Case Scope', other expected benefits are to assess the cost vs benefit value of having our own chatbot vs paying per user per month to use the NIH Enterprise versions of MS Copilot and OpenAI ChatGPT. This pilot will also be used to understand and develop a tiered business case on which NIAMS staff would fit which chatbot (NIAMS chatbot, MS copilot and OpenAI ChatGPT).","Inputs: Text input, text-based documents or PDF uploaded by users entered into a prompt. Amount or frequency of data are based on how often users use the chatbot, which will be determined during or after the pilot.  Outputs: Text summaries, information, recommendations and classifications. Output will be presented in text form. The frequency the AI produces the results is based on how often users use the chatbot.",Acquisition and/or Development,Neither,8/1/2024,8/1/2024,,,Developed with contracting resources.,47QTCA19D00A8,,,,,,,,No,,N/A,Documentation is missing or not available,,,,,,No,,Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, NICHD RPAB AI/ML Application Referral System,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"The NICHD RPAB AI/ML Application Referral System leverages the advanced AI/ML technologies to predict extramural branch assignments of new, incoming NICHD grant applications.  It employs human-in-the-loop development which enables human evaluation and offers the ability to override and alter model outputs.  Source data are from the NIH IMPAC II system and stored in the existing NICHD IT application CHIRP (Child Health Information Retrieval Program).  The AI process and output data have no impact on the NIH IMPAC II system. The business area addressed is the internal referral assignment performed by RPAB. The primary objective is to enhance the efficiency, accuracy, and consistency of grant application referral assignments, while reducing the burden on Subject Matter Experts in RPAB. The AI system is expected to streamline the process of internal referral of new grant applications.",Internal: NIH ImpacII and CHIRP results are presented as class predictions and class probabilities.,Implementation and Assessment,Neither,10/1/2023,10/1/2023,,,Developed in-house.,,,,,No,,Yes,No,Yes,,"NIH IMPAC II funded and unfunded grant application data is used. Unstructured text from project abstract, specific aims, and title are encoded and vectorized for model training and inference. Fiscal year, activity code, and RCDC terms are transformed via one-hot encoding for use in model training and inference. PII related to individuals associated with the grant is kept intact to preserve the integrity of the use case of grant application referral and the trends of researchers' focus on particular scientific areas.",Documentation is complete,,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, NLM-Chem: towards automatic chemical indexing in PubMed articles,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"NLM is the world’s largest biomedical library. MEDLINE is NLM's bibliographic database that contains references to journal articles in life sciences, with a concentration on biomedicine. Chemical indexing improves literature retrieval and information access. To assist this time-consuming and resource-intensive process, NLM developed NLM-Chem, an automatic tool for finding chemical names in the biomedical literature using advanced natural language processing and deep learning methods. ","Chemical indexing is part of the NLM's MEDLINE citation indexing efforts for improving literature retrieval and information access. Currently, chemicals indexing is performed manually by expert indexers. ",Initiated,Neither,9/1/2019,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, NLM-Gene: towards automatic gene indexing in PubMed articles,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"NLM is the world’s largest biomedical library. MEDLINE is NLM's bibliographic database that contains references to journal articles in life sciences, with a concentration on biomedicine. Gene indexing improves literature retrieval and information access. To assist this time-consuming and resource-intensive process, NLM developed NLM-Gene, an automatic tool for finding gene names in the biomedical literature using advanced natural language processing and deep learning methods. ","Gene indexing is part of the NLM's MEDLINE citation indexing efforts for improving literature retrieval and information access. Currently, gene indexing is performed manually by expert indexers.",Initiated,Neither,1/1/2020,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, NLP Automated Referral,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"Publicly reported under OER. NIGMS worked with eRA for over a year to incorporate natural language processing (NLP) and machine learning (ML) algorithms into eRA's Internal Referral Module (IRM).  IRM is now capable of using ML/NLP to assign applications directly to program officers (POs) based on the titles, abstracts, narratives, and specific aims of those applications, as well as the application histories of the PIs. The current process runs in the background on eRA's AWS cloud instances, and assigns the majority of new NIGMS applications to POs as soon as they are referred to NIGMS by CSR.  ML/NLP is also used to provide POs with the three most relevant NIH institutions to applications to facilitate IC fit determinations. Automated referral allows NIGMS to retain institutional referral knowledge by training on historical data, eliminates delays in referral by assigning applications as soon as they come in, and reduces burden on staff members and allows them to allocate more of their time to other high value tasks.","The tools uses IMPAC II application data, including titles, abstracts, narratives and specific aims to produce a top three of most relevant ICs and POs. ",Operation and Maintenance,Neither,8/1/2018,12/1/2019,8/1/2020,,Developed in-house.,,,,,No,,No,No,Yes,,All data come from the internal NIH IMPAC II database.,Documentation is missing or not available,,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,No,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, OCIO GenAI Advisor,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"The OCIO GenAI Advisor is a general productivity enhancement tool that uses Retrieval Augmented Generation (RAG) setup and user-provided data to answer questions about that data. This is a pilot project for experimentation to discover specific use cases, data needs, security and privacy concerns. If used effectively, the OCIO GenAI Advisor can shorten time to find the correct information and generate a non-trivial response for a complex question in a desired format such as security, architecture, PMO policy, reports.","Publicly available OMB, NIST policy in pdf format as user-provided data. Prompt and output are in natural language. ",Initiated,Neither,5/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, OIT Help Desk Chatbot,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"A general productivity enhancement tool that uses Retrieval Augmented Generation (RAG) setup to answer questions about that data. This is a proof of concept project for experimentation to discover specific use cases, data needs, security and privacy concerns. If used effectively, the OIT help desk can shorten resolution time for tickets, help empower users and generate a non-trivial response for a complex question in a desired format such as security, architecture, PMO policy, reports.","Publicly available help desk data, NIST policy in pdf format as user-provided data. Prompt and output are in natural language. ",Initiated,Neither,5/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Person-level disambiguation for PubMed authors and NIH grant applicants,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"The Office of Portfolio Analysis employs AI-based approaches to support analysis of the biomedical research landscape and inform data-driven decision-making by NIH leadership and extramural research administrators in other federal agencies. High-quality disambiguation is required to correctly link researchers to their grants and outputs including articles, patents, and clinical trials. ","The NIH Office of Portfolio Analysis developed a disambiguation solution that used article level metadata to assign 24.5M unique papers from the PubMed database to 16.0M unique author names, then used a novel neural network model to determine whether author-publication pairs refer to variant representations of the same person. ",Operation and Maintenance,Neither,2/1/2021,2/1/2021,2/1/2023,,Developed with contracting resources.,N/A,,,,No,,No,No,No,,N/A,Documentation is missing or not available,,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,More than 12 months,Yes,,Yes,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Portfolio Analysis Summarization Tool (PAST),Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"This AI Use Case is leveraging Power BI to present a dynamic view of grants within their portfolio, enabling program staff to explore their data interactively. Program staff face challenges in navigating and understanding extensive narratives within grants due to time constraints and the volume of information. The proposed Open AI sandbox will address these challenges by providing concise, AI-generated summaries of grant-related texts, including Titles, Abstracts, Specific Aims, and Research Strategies.   Rapid summarization of custom research portfolios which will be used to support program staff and others across the institute.","Input: Grants data from QVR. Output: Summaries of grant-related texts ",Initiated,Neither,10/1/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Program Classification Coding,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"A machine learning model has been developed to provide Program Classification Code (PCC) suggestions to assist NIA Division Staff with their work. The model is trained using IMPAC II data and generates a report of the top three PCC suggestions for a selected council date. Users can filter the data view by IC/OrgCode and Program Officer names to further customize the results. The Division staff is responsible for handling a large volume of applications during each council round, with the task of processing and assigning PCCs. To aid users, a solution has been developed that suggests the specific PCC that should be assigned to each application.","Model Inputs: Impac II data fields (Specific Aims, Project Title, and Study Section). Output is top 3 predicted PCCs. User views list of applications and top 3 suggestions by clicking on a report for the selected council date. User can filter view by IC/OrgCode and Program Officers names.  ",Initiated,Neither,10/1/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, RCDC AI Validation Tool,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"Annually, NIH reports the support level for various research, condition, and disease categories based on grants, contracts, and other funding mechanisms used across the agency. To provide the public the transparency into the reporting of NIH's funded research.",The goal of the tool is to ensure RCDC categories are accurate and complete for public reporting of data. ,Retired,Neither,,,,5/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Remediate Adobe .pdf documents to be more accessible,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"Manual curation and remediation of PDF files is time-consuming and can require substantial expertise to remediate previously published artifacts.   PDF documents should conformed to Section 508 accessibility standards. NLM is using AI to remediate Adobe .pdf files not currently accessible to Section 508 standards. ",PDF improved files are more accessible to readers who use assistive technology.,Initiated,Neither,10/1/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Research Performance Progress Report (RPPR) Report Comparison,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"The National Institute on Aging (NIA) handles a significant number of grant applications annually, requiring thorough review to prevent duplication, identify changes, and ensure optimal resource allocation. The current manual review process involves detailed comparisons across grant documents from different years, checking for overlaps with other projects, and summarizing extensive scientific text. With each Program Officer (PO) managing between 50 to 200+ grants and overseeing a total of $4 billion in grant funding, the process is highly time-consuming and prone to errors.  Streamline the NIA grant application review process.","Input: Grants data from QVR. Output: Identification of duplication and overlap in grants year over year.",Initiated,Neither,5/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Scientific summaries tool,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"The goal is to generate justifications for personnel actions including appointments, conversions, pay adjustments, and hiring/retention incentives for physicians, scientists, and veterinarians. Within NIAID DIR, we have team that drafts justifications for personnel actions based on the research being performed. This tool will be created to help them quickly and effectively prepare justifications for personnel actions for investigators in specific research fields.","Inputs: scientific publications, CV/Bib, BSC submissions and outcome memos, prior justifications, and, clinical protocols. Outputs: Scientific summary.",Implementation and Assessment,Neither,7/1/2024,7/1/2024,,,Developed in-house.,,,,,No,,No,No,No,,N/A,Documentation is widely available,,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Not Reported,Less than 6 months,Yes,,No,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Semantic group prediction in the UMLS Metathesaurus,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"Developing AI approaches (neural networks, heuristics) to predict the semantic groups of new terms to be added to the Unified Medical Language System (UMLS) Metathesaurus.   This is used during new concept integration into the UMLS Metathesaurus and is only intended to interact with the new concepts that will be added and existing UMLS Metathesaurus concepts.  There is no training or intended use of this product for sensitive data.  The hybrid system combining deep learning and heuristics should be able to effectively support UMLS editing and quality assurance.  Support UMLS Metathesaurus editing.  This should dramatically reduce the time needed to integrate new concepts into the UMLS Metathesaurus and increase the consistency. ","Input: UMLS Metathesaurus (new concepts). Output: Semantic group(s) for the new concept. Concepts are typically text words or phrases for example """"Lung Cancer"""", """"heart attack"""". This is used yearly when it is time to add new concepts to the UMLS Metathesaurus. ",Acquisition and/or Development,Neither,7/1/2021,11/1/2021,,,Developed with contracting resources.,GSA Contract Number: 47QTCA18D00JA,,,,,,,,Yes,,"UMLS Metathesaurus was used for development, training, and testing of the developed software. ",Documentation is missing or not available,,,,,,No,,Less than 6 months,Yes,,No,,Yes,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Similarity-based Application and Investigator Matching (SAIM),Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"NIGMS supports basic research that increases our understanding of biological processes and lays the foundation for advances in disease diagnosis, treatment, and prevention. NIGMS-funded scientists investigate how living systems work at a range of levels from molecules and cells to tissues and organs, in research organisms, humans, and populations. NIGMS needs to know how much total support an investigator has to ensure that it is not funding Principal Investigators who are already adequately resourced. The tool can determine if an application has significant overlap (Just-In-Time Other Support) with an application that is funded by another agency.",The Similarity-based Application and Investigator Matching (SAIM) system uses NLP to find non-NIH grants awarded to NIGMS Principal Investigators.,Operation and Maintenance,Neither,6/1/2023,6/1/2023,1/1/2024,,Developed in-house.,,,,,No,,No,No,Yes,,"Data for this project come from the internal NIH IMPAC II database, the publicly available NSF grants database, the publicly available SBIR grants database, and the publicly available HHS grants database, TAGGS.",Documentation is complete,,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, SingleCite: Improving single citation search in PubMed,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"NLM is the world’s largest biomedical library. SingleCite is search algorithm designed to improve single citation searches in the PubMed database. SingleCite helps increase the effectiveness of PubMed searches by making a user’s search for a specific document in PubMed more successful.","SingleCite predicts the probability of a retrieved document being the target of a query based on predefined variables. This search is important for scholarly databases, such as PubMed.",Operation and Maintenance,Neither,6/1/2017,6/1/2017,1/1/2023,,Developed in-house.,,,,,Yes,This use case is operated in compliance with HHS's agencywide Information Quality Act policy and procedures.,No,No,No,,PubMed literature collectionAn earlier tool that perfomed the same task,Documentation is complete,,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,No,,No,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, SRDMS NLP COI,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"NIAID administers grant portfolios in the extramural research program areas including Allergy, Immunology, & Transplantation, Infectious Diseases, and HIV/AIDS. Detect individuals who may pose a conflict-of-interest during the grant review process. ",The tool allows NIAID’s Scientific Review Program team to more easily identify conflicts of interest (COI) between grant reviewers and applicants using NLP methods.,Operation and Maintenance,Neither,1/1/2019,1/1/2019,1/1/2019,,Developed with contracting resources.,N/A,,,,No,,No,No,Yes,,Yes - Labeled dataset of grant applications and associated conflicts of interest is used to calculate pipeline evaluation metrics.,Documentation is complete,,,Yes,Yes – source code is publicly available.,http://tfs.niaid.nih.gov:8080/tfs/BI/SRP_COI/_git/srp_coi?path=%2F&version=GBmain&_a=contents,Yes,Not Reported,Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Stem Cell Auto Coder,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"NIGMS supports basic research that increases our understanding of biological processes and lays the foundation for advances in disease diagnosis, treatment, and prevention. NIGMS-funded scientists investigate how living systems work at a range of levels from molecules and cells to tissues and organs, in research organisms, humans, and populations. Manually coding Stem Cell Research subcategories is a time intensive process. NIGMS sped up this process by using automation.","The Stem Cell Auto Coder uses NLP and ML to predict the Stem Cell Research subcategories of an application: human embryonic, non-human embryonic, human induced pluripotent, non-human induced pluripotent, human non-embryonic, and non-human non-embryonic.",Implementation and Assessment,Neither,2/1/2023,2/1/2023,,,Developed in-house.,,,,,No,,No,No,Yes,,All data come from the internal NIH IMPAC II database.,Documentation is missing or not available,,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Study Section Clustering Tool (SSCT),Department of Health and Human Services,HHS,NIH,Health & Medical,,None of the above.,"This tool suggests how study sections might be organized into groups of similar scientific fields to decide how applications might be grouped into new study sections. Subject matter experts ultimately finalize the study section groupings.  This tool helps to ensure that applications are grouped into study sections in a way that maps on to the current state of the science.  ","Input: Text of grant applications. Output: Lists of study sections that should be grouped together for further review.  ",Operation and Maintenance,Neither,1/1/2022,1/1/2022,1/1/2023,,Developed with contracting resources.,N/A,,,,No,,No,No,Yes,,IMPACII,Documentation is complete,,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Center for Scientific Review  General Support System (CSR GSS) ,Less than 6 months,Yes,,No,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Synonymy prediction in the UMLS Metathesaurus,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"Developing artificial intelligence (AI) approaches (neural networks) to predict synonymy among biomedical concepts in the Unified Medical Language System (UMLS) Metathesaurus, with application to the insertion of new concepts into the UMLS Metathesaurus.  This is used during new concept integration into the UMLS Metathesaurus and is only intended to interact with the new concepts that will be added and existing UMLS Metathesaurus concepts.  There is no training or intended use of this product for sensitive data. Preliminary results indicate the performance of the system should be sufficient to support the insertion of new terms into the UMLS Metathesaurus.  Support UMLS Metathesaurus editing.  This should dramatically reduce the time needed to integrate new concepts into the UMLS Metathesaurus and increase the consistency. ","Input: UMLS Metathesaurus (new concepts). Output: Synonymy with existing Metathesaurus concepts. Concepts are typically text words or phrases for example """"Lung Cancer"""", """"heart attack"""". This is used yearly when it is time to add new concepts to the UMLS Metathesaurus.   ",Acquisition and/or Development,Neither,9/1/2019,3/1/2020,,,Developed with contracting resources.,GSA Contract Number: 47QTCA18D00JA,,,,,,,,Yes,,"UMLS Metathesaurus was used for development, training, and testing of the developed software. ",Documentation is missing or not available,,,,,,No,,Less than 6 months,Yes,,No,,Yes,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, TB Case Browser Image Text Detection,Department of Health and Human Services,HHS,NIH,Health & Medical,,None of the above.,"The NIAID TB Portals Program is a multi-national collaboration for tuberculosis (TB) data sharing and analysis to advance TB research. A tool that uses AWS Recognition managed AI service to detect text in images which could be potentially Personally Identifiable Information (PII)/ Protected Health Information (PHI) in TB Portals data. ",The tool detects and counts instances of text and if above a threshold throws a warning to the user who is uploading the image. User must then affirmatively bypass to upload.,Operation and Maintenance,Neither,1/1/2019,1/1/2019,1/1/2019,,Developed with contracting resources.,N/A,,,,No,,No,No,Yes,,"Yes, used to evaluate performance.",Documentation is complete,,,Yes,Yes – source code is publicly available.,https://github.com/niaid/tbportals_data_pii_check,Yes,Not Reported,Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Tech Support Chatbot,Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"It is designed to address two usage scenarios: (1) provide the ORS/ORF community with an interactive customer service chat interface for inquiries related to OIIT technology services, policy, and general information and (2) provide OIIT staff with the ability to receive answers in a variety of technical areas as they troubleshoot or manage efforts in their respective teams. The benefit of the AI is to improve administrative questions and efficiency of the information to the internal OIIT team.","Provides text, images and data. Input data source is local database. Frequency is TBD.",Initiated,Neither,6/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Tool for PO Lookup Assignment (TPAL),Department of Health and Human Services,HHS,NIH,Government Services (includes Benefits and Service Delivery),,None of the above.,"Similar to publicly reported IRM use case. The Tool for PO Assignment Lookup (TPAL) provides NIGMS Program staff with the ability to automatically search for appropriate Program Officers (POs), including their corresponding Program Area Codes (PACs), and NIH ICs to assign or reassign project proposals, or to provide advice to prospective applicants. TPAL uses natural language processing and machine learning to predict the three most relevant NIGMS POs, including their corresponding PACs, and the three most relevant NIH ICs from the project description/abstract text entered into a textbox. Note that because the PO and IC prediction algorithms are independent of each other, TPAL will always try to provide the most relevant program​ officers, even when NIGMS is not one of the top three recommended NIH ICs.   There are many occasions in which Division Directors, Branch Chiefs, and Program Officers wish to receive suggestions for the most appropriate people to talk to about a project proposal or where to send a proposal that might not be appropriate for NIGMS.","Input: Free form text in an online textbox.  Output: Top three most relevant ICs and POs and their probabilities. ",Operation and Maintenance,Neither,3/1/2020,3/1/2020,7/1/2020,,Developed in-house.,,,,,No,,No,No,Yes,,All data comes from the internal NIH IMPAC II database.,Documentation is complete,,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Transformative Research Award Anonymization Check (TRAAC),Department of Health and Human Services,HHS,NIH,Health & Medical,,None of the above.,"This tool helps to determine whether grant applications for NIH Director's Transformative Research Awards are properly anonymized by quickly detecting information that might identify the applicant in the Specific Aims or Research Strategy sections.  This tool automatically screens many pages and highlights any text that might allow identification of the applicants. The highlighted text is then examined by subject matter experts. This streamlines content screening and aids both NIH staff and the external scientific community in making more rapid and efficient determinations about whether anonymity has been compromised. ","Input: PDF files including text and images from two sections of the NIH grant applications. Output: Spreadsheets containing records of identified text/phrases as well as PDF files with text/phrases highlighted. ",Operation and Maintenance,Neither,1/1/2021,1/1/2021,1/1/2022,,Developed in-house.,,,,,Yes,This use case is operated in compliance with HHS's agencywide Information Quality Act policy and procedures.,No,No,Yes,,IMPACII ,Documentation is complete,,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Center for Scientific Review  General Support System (CSR GSS) ,Less than 6 months,Yes,,No,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Machine learning pipeline for mining citations from full-text scientific articles,Department of Health and Human Services,HHS,NIH,Health & Medical,,None of the above.,"The NIH Office of Portfolio Analysis developed a machine learning pipeline to identify scientific articles that are freely available on the internet and do not require an institutional library subscription to access. The pipeline harvests full-text pdfs, converts them to xml, and uses a Long Short-Term Memory (LSTM) recurrent neural network model that discriminates between reference text and other text in the scientific article. The LSTM-identified references are then passed through our Citation Resolution Service. For more information see the publication describing this pipeline: Hutchins et al 2019 (https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.3000385#sec003). Retired use case","The pipeline harvests full-text pdfs, converts them to xml, and uses a Long Short-Term Memory (LSTM) recurrent neural network model that discriminates between reference text and other text in the scientific article. The LSTM-identified references are then passed through our Citation Resolution Service.",Retired,Neither,,,,9/1/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Machine learning system to predict translational progress in biomedical research,Department of Health and Human Services,HHS,NIH,Health & Medical,,None of the above.,"A machine learning system that detects whether a research paper is likely to be cited by a future clinical trial or guideline. Translational progress in biomedicine can therefore be assessed and predicted in real time based on information conveyed by the scientific community’s early reaction to a paper. For more information see the publication describing this system: Hutchins et al 2019 (https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.3000416) Retired use case",A machine learning system that detects whether a research paper is likely to be cited by a future clinical trial or guideline. ,Retired,Neither,,,,9/1/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, "Assessing developing possible version of ""HHSGPT"" for OIDP-specific internal use",Department of Health and Human Services,HHS,OASH,Mission-Enabling (internal agency support),,None of the above.,"OIDP's HIV.gov: an exploration of possibly using a custom knowledge-based walled/isolated version of ""Chat GPT"" for our internal OIDP users to access information to help with internal operational efficiencies related to our work on national strategies. We have been meeting for almost 1 year with senior HHS officials working on AI to keep them up to date on our approach and efforts in both RAG LLM and generative AI. We describe our work as an internal library. The tool is called Ask ABE! (v3.0.0-alpha.1) - AI-Driven. Beneficial. Equitable (ABE). It's like having a digital librarian who can help us at OIDP easily access and use all the valuable resources from HIV.gov to support our work in ending new HIV transmissions! Our work is critical to ensuring the following issues are addressed: (1) Security: FedRAMP certified Azure Open AI 3.5 Turbo / 4o Global. (2) Privacy: Not tracking conversations nor usage. PII stripped. (3) Equity: HIV Language and HIV Language Guideline through internal system prompt. (4) Hallucinations: Internal system prompt - ABE HIV.gov knowledge base and conversation history. There is a low risk of this occurring compared to open systems like ChatGPT. (5) Risk: NIST AI Risk Management Framework: Govern, Map, Measure, Manage for Responsible AI, Observable AI, and Explainable AI (6) Certifications / Training / Support: Azure AI Certifications / Data Science DoJo / Corporate Thought Leadership Support: Responsible AI Framework.","OIDP internal federal staff would possibly be able to ask questions, develop content, and revise existing content using a unique dataset of OIDP information. Inclusion of NIH HIV guideline is embedded in the internal prompting. ",Initiated,Neither,9/1/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Informal generative AI research for OIDP's HIV.gov,Department of Health and Human Services,HHS,OASH,Education & Workforce,,None of the above.,"OIDP's HIV.gov monitors (1) new and emerging AI tools/policies, generative AI use in the field of communications, and (2) the impact of generative AI on HIV federal content delivery through major search engines. This work is done by an ORISE fellow in conjunction with the HIV.gov communications team. The information is synthesized and discussed during a weekly communications meeting to determine any possible application or if additional follow-up is needed. The purpose of this activity is to determine potential tools that we may be able to use to extend the reach of our work. For example, we have been monitoring tools that potentially can synthesize weekly content into a short narrated video.",This practice helps us guide our planning.,Implementation and Assessment,Neither,11/1/2022,11/1/2022,,,Developed with both contracting and in-house resources.,N/A,,,,No,,No,No,No,,N/A,Documentation has been partially completed,,,No,No – agency does not have access to source code.,,No,,Less than 6 months,No,,No,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Presenting/providing technical assistance on HIV.gov's cautious & transparent use of generative AI,Department of Health and Human Services,HHS,OASH,Education & Workforce,,None of the above.,"OIDP's HIV.gov hosts generative AI workshops, 1-on-1 trainings, and limited generative AI labs to share our cautious and transparent use of generative AI to create communications content. These activities also include information on generative AI basics and information on federal guidance. Our limited AI-generated content is disclosed with a watermark or a tag saying, ""Created by AI, Reviewed By Humans."" These trainings have occurred at 5 national HIV-related conferences and include multiple 1-1 training sessions and workshops that reinforce the importance of following best practices, being transparent, and being cautious. We have trained over 500 people on the cautious and transparent use of generative AI.","These offerings help our audiences understand the importance of cautious and transparent use, federal resources, and additionally promote information about our office's HIV programs.",Operation and Maintenance,Neither,3/1/2023,3/1/2023,3/1/2023,,Developed with both contracting and in-house resources.,N/A,,,,No,,No,No,No,,N/A,Documentation is widely available,,,No,No – agency does not have access to source code.,,No,,Less than 6 months,No,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, Utilizing HHSGPT Pilot for writing/editing ,Department of Health and Human Services,HHS,OASH,Mission-Enabling (internal agency support),,None of the above.,"Limited OIDP staff have been selected to have access to HHSGPT, an internal generative AI tool in its pilot phase. Team members have been using it for editing, brainstorming, idea generating, drafting text, revisions, gap analysis, organization/structure, evaluations, etc. HHSGPT has been used to assess national strategies, such as the 2026-2030 vaccine national strategy, the 2026-2030 viral hepatitis strategy, and others. When utilizing HHSGPT, we work to report back how this use has reduced labor burden and note any interesting findings when using the tool. Use reporting is done on an Excel file. ","HHSGPT uses a closed tool to generate suggestions, responses, edits, and information that may be useful for OIDP tasks related to infectious diseases or HIV information/policies. Team members have been using it for editing, brainstorming, idea generating, drafting text, revisions, gap analysis, organization/structure, evaluations, etc. ",Operation and Maintenance,Neither,10/1/2023,10/1/2023,10/1/2023,,Developed in-house.,,,,,No,,No,No,No,,HHS,Documentation is complete,,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Not Reported,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, HHSGPT,Department of Health and Human Services,HHS,OCIO,Government Services (includes Benefits and Service Delivery),,None of the above.,"ChatGPT is an AI-powered chatbot developed by OpenAI, and it can be integrated with Azure, Microsoft's cloud computing platform. ChatGPT leverages the power of OpenAI's language model to provide natural language understanding and generation capabilities. Scalability: As an AI-powered chatbot, ChatGPT can handle multiple conversations simultaneously, scaling up to meet high demand without the need for additional human resources. Increased Efficiency: ChatGPT can automate repetitive tasks, freeing up human agents to focus on more complex or value-added activities, improving overall efficiency. Persona Benefits: Assist data scientists in data exploration, model prototyping, analysis, collaboration, and accessing relevant knowledge, enhancing their productivity and efficiency. Assist policy analysts by providing instant insights, answering complex policy-related questions, and facilitating interactive discussions, enabling faster and more informed decision-making. Assist in writing large policy documents by providing suggestions, generating coherent content, and offering language refinement, streamlining the writing process and ensuring high-quality outputs.","At present, HHSGPT is undergoing a pilot phase throughout the entire HHS organization under an IATT.",Acquisition and/or Development,Neither,9/1/2023,9/1/2023,,,Developed in-house.,,,,,,,,,No,,None,Documentation has been partially completed,,,,,,Yes,HHSGPT,Less than 6 months,Yes,,No,,No,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, DECIDE,Department of Health and Human Services,HHS,SAMHSA,Health & Medical,,None of the above.,"At present, SAMHSA's Office of Behavioral Health Equity (OBHE) manually reviews grantee organizations' progress towards achieving their health disparity reduction goals. This process involves manually extracting Disparity Impact Statements, annual reports, enrollment data, and external population data to verify the accuracy of grantees' reported progress. This is a time-consuming process, thus the development of an AI-based tool for doing this review is in development. DECIDE will reduce the administrative burden on OBHE staff and allow them to provide feedback to grantees more quickly.","DECIDE will use generative AI to automate the process of extracting data from various sources, then comparing across data sources to determine (a) whether a grantee has accurately reported the demographic data of the population they serve and (b) how much progress they have made towards decreasing health disparities for the vulnerable population they have identified.",Initiated,Neither,6/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Document drafting and editing,Department of Health and Human Services,HHS,SAMHSA,Mission-Enabling (internal agency support),,None of the above.,"SAMHSA employees routinely write internal reports, public-facing written products, and other documents. Some parts of the writing process, such as structuring thoughts into an outline and proofreading, can be enhanced through the use of AI. The document drafting and editing tool will increase employee productivity and help produce polished written work products.","The document drafting and editing tool will serve as a writing assistant to help SAMHSA employees present their ideas clearly and effectively. It will function similarly to a standard large language model chatbot, allowing the user to query the model to perform specific writing tasks, such as outlining and proofreading.",Initiated,Neither,8/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Document summarization,Department of Health and Human Services,HHS,SAMHSA,Mission-Enabling (internal agency support),,None of the above.,"SAMHSA employees routinely read and summarize large amounts of documentation for grantmaking, management, and other day-to-day work. Ingesting and condensing such vast amounts of information can take time away from SAMHSA's most important mission-driven work. The document summarization tool will expedite day-to-day workflows and help employees ingest large amounts of information more quickly.","The document summarization tool will allow users to upload documents and produce a summary of these documents. The tool will have enhanced data privacy mechanisms. Additionally, the tool will not use any SAMHSA-provided input as training data, ensuring that private information stays private to SAMHSA.",Initiated,Neither,8/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Smart Search,Department of Health and Human Services,HHS,SAMHSA,Government Services (includes Benefits and Service Delivery),,None of the above.,"SAMHSA.gov team would like to procure ""Kendra"" to implement a smart search on store.samhsa.gov—improving access to the Federal Government’s leading source of behavioral health information. Currently the search on store.samhsa.gov relies on key words pulled from document titles, descriptions, and manually generated tags. Both anecdotal evidence and results of usability testing have demonstrated that the Store’s search capability is the single greatest barrier to users accessing materials they need Kendra will allow for scanning of all documents in the store to respond more accurately to consumer queries while providing a faster search. ","Kendra is not generative AI and does not provide AI-generated responses to queries. Kendra operates within strict guardrails to ensure the security and privacy of information, particularly concerning the handling of sensitive data like Personally Identifiable Information (PII).",Initiated,Neither,9/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Counterparty Risk Anomaly Detection ,Department of Housing and Urban Development,HUD,Ginnie Mae ,Mission-Enabling,,None of the above.,"Ginnie Mae is responsible for analyzing counterparty risk profiles of mortgage issuers who participate in Ginnie Mae’s program. Ginnie Mae analyzes data from multiple sources to identify potential risks and areas of focus. To enhance the identification of data patterns, Ginnie Mae uses machine learning algorithms, specifically clustering and genetic techniques. These algorithms detect potential risk areas, enabling a focused approach to subsequent analysis by Ginnie Mae staff. ",Data patterns ,Operation and Maintenance,Neither,8/1/2020,8/1/2020,5/1/2022,N/A ,Developed with contracting resources.,GSQ1117BJ0043 ,No ,,,No ,"Ginnie Mae has established Data Certification policies with standards for accuracy, completeness, and reasonableness. Ginnie Mae also follows HUD’s Project Planning & Management discipline that establishes requirements for data quality and management. ",No,Yes,Yes,,"This is not a self-learning system. Although the data is not used to train, fine-tune, and/or evaluate performance, the following sources are used by the ML model to perform analysis: - GNMA Investor Reported MBS portfolio data aggregated on an issuer level - MBFRF (Mortgage Banking Financial Reporting Form) Data for counterparty financial information ","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Ginnie Mae Reporting and Feedback System (RFS) ,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.","No - Use Case is not Rights or Safety impacting, section 5 does not apply",,,,,,,,,,,,,,,,, Subledger Data Quality Machine Learning,Department of Housing and Urban Development,HUD,Ginnie Mae ,Mission-Enabling,,None of the above.,"Ginnie Mae analyzes Master Sub-Servicer (MSS) transaction data on a monthly cadence. Through its use of machine learning models, Ginnie Mae has enhanced its ability to identify data inconsistencies and exceptions associated with its MSS transaction data. By automating the analysis of a significant number of MSS transactions, the machine learning algorithms enhance the efficiency and accuracy of key reporting processes. ",The AI solution identifies data anomalies. ,Operation and Maintenance,Neither,11/20/2020,11/20/2020,4/21/2021,N/A ,Developed with contracting resources.,GS11Q16BJC0005 ,No ,,,No ,"Ginnie Mae has established Data Certification policies with standards for accuracy, completeness, and reasonableness.  Ginnie Mae also follows HUD’s Project Planning & Management discipline that establishes requirements for data quality and management. ",No,Yes,No,,"This is not a self-learning or self-refining system. Although the data is not used to train, fine-tune, and/or evaluate performance, the following sources are used: - Transaction data from Ginnie Mae Master Sub-Servicers for the defaulted Single-Family non-pooled assets (this data does not include any PII). ","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Ginnie Mae Financial Accounting System  ,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.","No - Use Case is not Rights or Safety impacting, section 5 does not apply",,,,,,,,,,,,,,,,, Automating Draft Counterparty Credit Narrative Reports,Department of Housing and Urban Development,HUD,Ginnie Mae ,Mission-Enabling,,None of the above.,"Ginnie Mae performs counterparty credit reviews of mortgage issuers who participate in Ginnie Mae’s program. The reviews are based on written analysis and financial data. As an initial step in counterparty credit reviews, Ginnie Mae uses Natural Language Generation (NLG) to implement coded rules and generate draft narratives. This application of AI enables processing efficiency and reduces errors that can occur in a manual process. ",Draft memos/reports ,Operation and Maintenance,Neither,2/21/2021,3/1/2021,4/1/2022,N/A ,Developed with contracting resources.,47QFDA19A0008 ,No ,,,No ,"Ginnie Mae has established Data Certification policies with standards for accuracy, completeness, and reasonableness.  Ginnie Mae also follows HUD’s Project Planning & Management discipline that establishes requirements for data quality and management. ",No,Yes,Yes,,"This is not a self-learning or self-refining system. The reports are generated from business rules and format established by Ginnie Mae staff. Although the data is not used to train, fine-tune, and/or evaluate performance, the following sources are used: -GNMA Investor Reported MBS portfolio data aggregated on an issuer level - MBFRF (Mortgage Banking Financial Reporting Form) Data for counterparty financial information ","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Ginnie Mae Reporting and Feedback System (RFS) ,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.","No - Use Case is not Rights or Safety impacting, section 5 does not apply",,,,,,,,,,,,,,,,, Voice of the Customer,Department of Housing and Urban Development,HUD,Office of the Chief Financial Officer (OCFO),Mission-Enabling,,None of the above.,"The Office of the Chief Financial Officer, Customer Experience Team works across the department to develop a deep understanding of who our customers are and how we can best serve their needs. The Voice of the Customer application brings the best-in-class voice transcription, speech, and text analytics to customer feedback surveys, contact center calls and chats. It allows HUD to unlock critical insights that provide deeper understanding of how to support and manage our program/service delivery and contact center providers, and ultimately to improve customers’ experiences.","A data analytics dashboard that transforms customer feedback into succinct, easy-to-understand metrics, such as customer satisfaction rates across different customer interaction and call center topics.",Operation and Maintenance,Neither,1/15/2024,9/30/2022,8/30/2024,N/A ,Developed with contracting resources.,86614922C00009,No,,,No,N/A,Yes,Yes,Other,This AI use case is part of a larger system that collects data directly from HUD customers,"This system allows HUD to build surveys of varying complexity, distribute surveys via different delivery methods, centrally manage customer feedback data, and enable enterprise-wide qualitative and quantitative data analysis on both data collected through the system and data imported into the system. The Voice of the Customer data can come in through a variety of ways. It can be a web/paper survey, a phone call, an email, or anything having all that separate data on one platform will help OCFO’s Customer Experience team hear what customers have to say holistically. The gathered feedback is both customer and employee experience data as it pertains to their interactions with HUD. Contact information will be requested, but is voluntary/optional, and is only collected/used for future research. This would include gathering a person’s name and contact information they are willing to provide (email or phone number). This data would be used for research purposes only to further investigate how to improve customers’ experiences with HUD. These interactions would include moments when the customer/employee interacts with HUD (For example, a customer visits the HUD website for information, or an employee visits the intranet for information). The data would be collected via surveys or conversations regarding their experience within that specific moment and/or overall experience.","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,No,No – agency does not have access to source code.,,Yes,P351 Voice of the Customer (VoC),Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.","No - Use Case is not Rights or Safety impacting, section 5 does not apply",,,,,,,,,,,,,,,,, Quantitative Text Analysis,Department of Housing and Urban Development,HUD,Office of the Chief Financial Officer (OCFO),Mission-Enabling,,Summarizing the key points of a lengthy report using AI.,"Analysis of large amounts of text for patterns (commonly appearing words, common word clusters, etc.)","The outputs are charts and graphs showing which words appear most commonly across the documents, which words appear clustered together most often, and the sentiment of the words, etc. ",Retired,Neither,11/1/2023,11/1/2023,12/1/2023,1/1/2024,Developed with contracting resources.,"GS-00F-275CA-86615121A00004, TO 86615122F0020",No,,,No,N/A,No,Yes,No,,None,N/A,None,,No,N/A,,N/A,Leveraged Audient Group contract service to produce output,N/A,N/A,,N/A,,N/A,,N/A,N/A,"No - Use Case is not Rights or Safety impacting, section 5 does not apply",,,,,,,,,,,,,,,,, Translation of Digital Media,Department of Housing and Urban Development,HUD,Office of Public Affairs (OPA),Mission-Enabling,,None of the above.,"OPA is using Google Translate to translate information on public-facing websites. On every HUD.gov webpage, there is an “Español” button that calls Google Translate to translate the page into Spanish. From there, a Google Translate banner appears on the page allowing for the translation into any available language. This is a free, off-the-shelf solution with no customization. It is also an interim solution to improve language access at HUD while a permanent solution is being developed in line with HUD’s Language Access Plan in response to Executive Order 13166. ",Translated content on HUD.gov,Operation and Maintenance,Both,5/1/2024,5/1/2024,5/1/2024,N/A ,Developed in-house.,N/A,No,,,Yes,N/A,No,Yes,Other,The data set for this use case is all the public-facing content available on HUD.gov,None,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,No,No – agency does not have access to source code.,,No,Google Translate,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,Yes,Agency CAIO has waived this minimum practice and reported such waiver to OMB.,"The Office of Public Affairs completed an AI Impact Assessment and identified the following risks for machine translation of digital media: - There are known challenges with AI translation when it comes to fully understanding cultural backgrounds and nuances (for example, translating idioms, proverbs, humor, colloquialisms, slang, and sarcasm) and this can result in awkward translations. - Text submitted to the Google Translate API is sent to Google's servers for processing. - Initially there was data to suggest that there was gender bias evident in translations, specifically by mostly defaulting to male translations.",Agency CAIO has waived this minimum practice and reported such waiver to OMB.,No monitoring protocols have been established: Necessary infrastructure needed to perform monitoring of AI systems in production is not available and/or a plan to monitor models has not yet been established.,"Other – Immediate human intervention is not practicable; however, an appropriate fail-safe has been implemented. ",There are no decisions or actions associated with this use case. It simply translates content available on HUD.gov.,Agency CAIO has waived this minimum practice and reported such waiver to OMB.,,No,Agency CAIO has waived this minimum practice and reported such waiver to OMB.,Agency CAIO has waived this minimum practice and reported such waiver to OMB.,,Agency CAIO has waived this minimum practice and reported such waiver to OMB.,,Agency CAIO has waived this minimum practice and reported such waiver to OMB., NextGen Advanced Methods: Air Traffic Control System Command Center (ATCSCC) Webinar Speech2Text and Analysis,National Aeronautics and Space Administration,NASA,Ames Research Center,Natural Language Processing,None,None of the above.,"The Advanced Methods project explores the use of innovative and emerging technologies to drive post operational analysis of Traffic Management for aircraft. Technologies such as machine learning, (ML), artificial intelligence (AI), and advanced data analytics for use in improving the FAA's traffic flow management. In this specific use case, our aim is to use deep learning to convert live ATCSCC webinar meeting conversation to text, and then apply natural language processing to the converted text data for later analysis and review.",Text from speech to text and NLP of air traffic management content.,In production,Neither,1/1/2021,1/1/2021,NA,NA,Developed with both contracting and in-house resources.,NA,No,NA,None,No,NASA's data stewardship best practices are in effect under guidance from the Chief Data Officer.,No,No,Yes,Blank/Not Provided,None,Yes,N/A,,Yes,Yes,NA,Operated in an approved enclave,ATCSCC,Less than 6 months,No,,Yes,,,,This project leveraged external NLP work,Yes,,,,,,,,,,,,,,,,,, NextGen Data Analytics: Letters of Agreement,National Aeronautics and Space Administration,NASA,Ames Research Center,Natural Language Processing,None,None of the above.,"Today, operation constraints are documented via Standard Operating Procedures (SOP) and Letters of Agreement (LOA) and are not made available to the public in a consistent manner. SOPs are specific to an air traffic control facility and specify the procedures necessary for safe operation in the sector. LOA outline agreements establishing procedures and responsibilities between two parties (including crossing restrictions, holding patters, emergency procedure coordination, etc.) The LOA/SOPs are published internally as scanned PDFs and are the responsibility of the facility to maintain. To reduce the manual effort of tagging the documents for ease of reference, there is an opportunity to use modern data analytics and machine learning to produce and disseminate constraints in a standardized manner.  Providing LOAs or SOPs to stakeholders will enable flight planners (pilots and vendors) to study or ingest this information and thereby plan flight trajectories that remain consistent with air traffic constraints. It is also fundamental to Next Gen capabilities to share accurate data for purposes of creating new noise abatement procedures; improve NAS information for common situational awareness and alignment to implement new tools to assist in future time-based flow management.",Test-based digitized SOPs and LOAs.,In production,Neither,1/1/2021,1/1/2021,NA,NA,Developed with both contracting and in-house resources.,NA,No,NA,None,No,NASA's data stewardship best practices are in effect under guidance from the Chief Data Officer.,No,No,Yes,Blank/Not Provided,Letter of agreement dataset from FAA,Yes,N/A,,Yes,Yes,NA,Operated in an approved enclave,Checking,Less than 6 months,No,,Yes,,,,This project used external text mining capabilities,Yes,,,,,,,,,,,,,,,,,, AEGIS: Autonomous Exploration for Gathering Increased Science,National Aeronautics and Space Administration,NASA,Jet Propulsion Laboratory,Deep Learning,None,None of the above.,"AEGIS enables intelligent targeting and data acquisition by planetary rovers. It uses computer vision techniques to identify targets (e.g., rocks) in wide angles images of the rover's surrounding terrain. If targets are found that match scientists specificiations, they are then measured autonomously using remote sensing instruments. AEGIS was first used on the MER Mission. It is currently in use on the MSL Mission to acquire data for the ChemCam instrument. It is planned for use in Spring of 2022 on the M2020 Mission to acquire data for the SuperCam instrument.","Recommendations of relevant objects, e.g., Mars rocks, for scientific examination.",In production,Neither,1/1/2006,1/1/2007,1/1/2010,NA,Developed with both contracting and in-house resources.,NA,No,NA,None,No,NA,No,No,N/A,Blank/Not Provided,public Mars science data related to sensors for optimizing scientific measurements,NA,N/A,,Yes,Yes,NA,No. Engineering Review and Resease instead.,Mars2020 Rover,N/A,N/A,,N/A,,,,Mars 2020 team built on prior rovers and shared across different facets of Mars 2020.,Yes. Peer review,,,,,,,,,,,,,,,,,, ASPEN Mission Planner,National Aeronautics and Space Administration,NASA,Jet Propulsion Laboratory,Statistical Methods,None,None of the above.,"Based on AI techniques, ASPEN is a modular, reconfigurable application framework which is capable of supporting a wide variety of planning and scheduling applications. ASPEN provides a set of reusable software components that implement the elements commonly found in complex planning/scheduling systems, including: an expressive modeling language, a resource management system, a temporal reasoning system, and a graphical interface. ASPEN has been used for many space missions including: Modified Antarctic Mapping Mission, Orbital Express, Earth Observing One, and ESA's Rosetta Orbitter.",Plan and schedule recommendations to optimize Science from Scientific Sensors,In mission,Neither,1/1/1997,1/1/1998,1/1/2002,NA,Developed with both contracting and in-house resources.,NA,No,NA,None,No,NASA's data stewardship best practices are in effect under guidance from the Chief Data Officer.,No,No,Yes,Blank/Not Provided,"Data related to optimizing sensors and flight hardware. Note This is a mission-embedded software component subject to system engineering and mission planning reviews, not the IT ATO process. PII, SAOP, HISP, etc. do not apply.",Yes,N/A,,Yes,Yes,NA,No. Engineering Review and Resease instead.,Multiple approved space-based systems,N/A,No,,N/A,,,,This capability has been re-used among many missions,Yes. Peer review,,,,,,,,,,,,,,,,,, CLASP Coverage Planning & Scheduling,National Aeronautics and Space Administration,NASA,Jet Propulsion Laboratory,Statistical Methods,None,None of the above.,"The Compressed Large-scale Activity Scheduling and Planning (CLASP) project is a long-range scheduler for space-based or aerial instruments that can be modeled as pushbrooms 1D line sensors dragged across the surface of the body being observed. It addresses the problem of choosing the orientation and on/off times of a pushbroom instrument or collection of pushbroom instruments such that the schedule covers as many target points as possible, but without oversubscribing memory and energy. Orientation and time of observation is derived from geometric computations that CLASP performs using the SPICE ephemeris toolkit. CLASP allows mission planning teams to start with a baseline mission concept and simulate the mission's science return using models of science observations, spacecraft operations, downlink, and spacecraft trajectory. This analysis can then be folded back into many aspects of mission design -- including trajectory, spacecraft design, operations concept, and downlink concept. The long planning horizons allow this analysis to span an entire mission. Actively in use for optimized scheduling for the NISAR Mission, ECOSTRESS mission (study of water needs for plant areas), EMIT mission (mineralogy of arid dusty regions), OCO-3 (atmospheric CO2) and more as well as used for numerous missions analysis and studies (e.g. 100+).","Estimates of scientific mission outcomes / results, based on optimized scheduling of spacecraft and sensors.",In mission,Neither,1/1/2006,1/1/2006,1/1/2018,NA,Developed with both contracting and in-house resources.,NA,No,NA,None,No,NASA's data stewardship best practices are in effect under guidance from the Chief Data Officer.,No,No,Yes,Blank/Not Provided,Mission planning and execution data from multiple missions.,Yes,N/A,,Yes,Yes,NA,No. Engineering Review and Resease instead.,Deployed on multiple remote sensing spacecraft,,No,,N/A,,,,This capability has been re-used among many missions,Yes,,,,,,,,,,,,,,,,,, Enhanced AutoNav for Perseverance Rover on Mars,National Aeronautics and Space Administration,NASA,Jet Propulsion Laboratory,Classification,None,None of the above.,"AutoNav on the Perseverance Rover autonomously plans a safe path based on stereo navigation camera images, based on multiple technologies including a tree search for decision making, Dijkstra algorithm for global path planning, stereo processing for 3D terrain reconstruction, and Approximate Clearance Evaluation (ACE) for safety checks. It is deployed on Perserverance rover and being used for autonomous driving on Mars.",Recommended navigation path for Mars Rover.,In production,Neither,1/1/2015,1/1/2018,7/1/2020,NA,Developed with both contracting and in-house resources.,NA,No,NA,None,No,NASA's data stewardship best practices are in effect under guidance from the Chief Data Officer.,No,No,Yes,Blank/Not Provided,Sensor and imagery data from prior Mars missions and from terrestrial-based physical simulation sites.,Yes,N/A,,Yes,Yes,NA,No. Engineering Review and Resease instead.,Mars2020 Rover,Less than 6 months,No,,N/A,,,,Reused selected techniques from prior Mars rovers,Yes,,,,,,,,,,,,,,,,,, Mars2020 Rover (Perseverance),National Aeronautics and Space Administration,NASA,Jet Propulsion Laboratory,Other,None,None of the above.,"Research, experiments, and engineering to empower future rovers with onboard autonomy; planning, scheduling & execution; path planning; onboard science; image processing; terrain classification; fault diagnosis; and location estimation. This is a multi-faceted effort and includes experimentation and demonstrations on-site at JPL's simulated mars navigation yard. The M2020 onboard scheduler incrementally constructs a feasible schedule by iterating through activities in priority-first order. When considering each activity it computes the valid time intervals for placement, taking into account preheating, maintenance heating, and wake/sleep of the rover as required. After an activity is placed (other than a preheat/maintenance or wake/sleep), the activity is never reconsidered by the scheduler for deletion or moving. Therefore the scheduler can be considered non backtracking, and only searches in the sense that it computes valid timeline intervals for legal activity placement. Meta Search: Because the onboard scheduler will be invoked many times in a given sol (Martian Day) with a range of possible contexts (due to execution variations), its non backtracking nature leaves its vulnerable to brittleness. In order to mitigate this potential brittleness, the Copilot systems perform a monte carlo based stochastic analysis to set meta parameters of the scheduler - primarily activity priority but also potentially preferred time and temporal constraints.",Mission-priority-based recommendations for scheduling Mars2020 Rover activities.,In production,Neither,1/1/2015,1/1/2018,7/1/2020,NA,Developed with both contracting and in-house resources.,NA,No,NA,None,No,NASA's data stewardship best practices are in effect under guidance from the Chief Data Officer.,No,No,Yes,Blank/Not Provided,Multiple forms of rover mission data,Yes,N/A,,Yes,Yes,NA,No. Engineering Review and Resease instead.,Mars2020 Rover,Less than 6 months,No,,no,,,,Reused selected techniques from prior Mars rovers,Yes,,,,,,,,,,,,,,,,,, MLNav (Machine Learning Navigation),National Aeronautics and Space Administration,NASA,Jet Propulsion Laboratory,Deep Learning,None,None of the above.,"Accelerates path planning of rovers and other types of vehicles through ML-based heuristics, while guaranteeing safety through conventional, model-based collision checking. Integrated with M2020's Enhanced AutoNav (ENav) and tested with the real terrain data from Mars on ENav simulator. Uses U-net for the ML-based heuristics, trained by simulation-generated terrain data.",Path planning recommendations for Mars2020 Rover,In production,Neither,10/1/2019,9/30/2021,7/1/2020,NA,Developed with both contracting and in-house resources.,NA,No,NA,None,No,NASA's data stewardship best practices are in effect under guidance from the Chief Data Officer.,No,No,Yes,Blank/Not Provided,Simulated terrain data for Mars,Yes,N/A,,Yes,Yes,NA,No. Engineering Review and Resease instead.,Mars2020 Rover,Less than 6 months,No,,No,,,,No,Yes,,,,,,,,,,,,,,,,,, Perseverance Rover on Mars - Terrain Relative Navigation,National Aeronautics and Space Administration,NASA,Jet Propulsion Laboratory,Deep Learning,None,None of the above.,"3D machine vision via dual cameras to inform convolutional neural networks for rover navigation path planning. Physics / momentum - based Terrain Relative Navigation (TRN). Enhanced AutoNavigation (Enav) plans feasible paths using multiple techniques, to include random forests. Approximate Clearance Evaluation (ACE) assesses obstacles to deterimine if the suspension of the rover can clear them (drive over them) or needs to route around them. Additional, more complex AIML techniques are being planned and tested for future rovers at JPL. TRN was also used to provide precision landing for the Perseverance Rover in the entry, descent, and landing process to get the rover to the surface of Mars.",Real-time terrain-relative navigation recommendations.,In production,Neither,1/1/2004,10/1/2014,2/18/2021,NA,Developed with both contracting and in-house resources.,NA,No,NA,None,No,NASA's data stewardship best practices are in effect under guidance from the Chief Data Officer.,No,No,Yes,Blank/Not Provided,Simulated terrain data for Mars,Yes,N/A,,Yes,Yes,NA,No. Engineering Review and Resease instead.,Mars2020 Rover,Less than 6 months,No,,No,,,,Built on prior TRN work,Yes,,,,,,,,,,,,,,,,,, SPOC (Soil Property and Object Classification),National Aeronautics and Space Administration,NASA,Jet Propulsion Laboratory,Deep Learning,None,None of the above.,"Using a convolutional neural network (CNN), SPOC (Soil Property and Object Classification) takes rover images and classifies the terrain type (e.g., sand, soil) from visual appearance. This ability enables rover to drive more safely. It is trained by labeled images from MER (Mars Exploration Rover), MSL (Mars Science Laboratory), and Mars 2020 rovers, annotated by tens of thousands of citizen scientsts through the AI4Mars project. SPOC deployed on MSL's ground operation system and onboard test on M2020 is being considered. SPOC is one of many inputs to navigation.","Terrain image classifications such as soil, sand, rock, etc.",In production,Neither,1/1/2016,1/1/2022,7/1/2020,NA,Developed with both contracting and in-house resources.,NA,No,NA,None,No,NASA's data stewardship best practices are in effect under guidance from the Chief Data Officer.,No,No,Yes,Blank/Not Provided,"As an embedded system on the Mars Rover, ATO, PII, SAOP, HISP, etc. do not apply",Yes,N/A,,Yes,Yes,https://data.nasa.gov/Space-Science/AI4MARS-A-Dataset-for-Terrain-Aware-Autonomous-Dri/cykx-2qix,No. Engineering Review and Resease instead.,Multiple approved space-based systems,Less than 1 Year,No,,Yes,,,,Reused for many scientific missions,Yes,,,,,,,,,,,,,,,,,, Volcano SensorWeb,National Aeronautics and Space Administration,NASA,Jet Propulsion Laboratory,AIML Platform/Environment,None,None of the above.,"The Sensor Web Project uses a network of sensors linked by software and the internet to an autonomous satellite observation response capability. This system of systems is designed with a flexible, modular, architecture to facilitate expansion in sensors, customization of trigger conditions, and customization of responses. This system has been used to implement a global surveillance program to study volcanos. We have also run sensorweb tests to study flooding, cryosphere events, and atmospheric phenomena. Specifically, in our application, we use low resolution, high coverage sensors to trigger observations by high resolution instruments. Note that there are many other rationales to network sensors into a sensorweb. For example automated response might enable observation using complementary instruments such as imaging radar, infra-red, visible, etc. Or automated response might be used to apply more assets to increase the frequency of observation to improve the temporal resolution of available data. Our sensorweb project is being used to monitor the Earth's 50 most active volcanos. We have also run sensorweb experiments to monitor flooding, wildfires, and cryospheric events (snowfall and melt, lake freezing and thawing, sea ice formation and breakup.)",Detection and classification of volcano-related features.,In production,Neither,1/1/2003,1/1/2003,1/1/2003,NA,Developed with both contracting and in-house resources.,NA,No,NA,None,No,NASA's data stewardship best practices are in effect under guidance from the Chief Data Officer.,No,No,Yes,Blank/Not Provided,Terrestrial and space-based remote sensing satellite images to train the model,Yes,N/A,,Yes,Yes,NA,No. Engineering Review and Resease instead.,Deployed on multiple remote sensing spacecraft,Less than 6 months,No,,Yes,,,,This capability has been re-used among many missions,Yes,,,,,,,,,,,,,,,,,, NASA OCIO STI Concept Tagging Service,National Aeronautics and Space Administration,NASA,Langley Research Center,Natural Language Processing,None,None of the above.,An API (application program interface) for exposing topic models created with the STI (Scientific & Technical Information) concept training repository.,API standards,In production,Neither,1/1/2021,1/1/2021,NA,NA,Developed with both contracting and in-house resources.,NA,No,NA,None,No,NASA's data stewardship best practices are in effect under guidance from the Chief Data Officer.,No,No,Yes,Blank/Not Provided,,Yes,N/A,,Yes,Yes,NA,No. Engineering Review and Resease instead.,STI,Less than 6 months,No,,Yes,,,,This project re-used open source and prior NLP work,Yes,,,,,,,,,,,,,,,,,, Autonomous WAiting Room Evaluation (AWARE),National Aeronautics and Space Administration,NASA,Langley Research Center,Deep Learning,None,None of the above.,"Using an existing security camera and YOLO Machine Learning model to detect and count number of people waiting for service at Langley's Badge & Pass Office. When a predetermined threshold of people is exceeded, automated texts and emails are sent to request additional help at the service counters.",Count of current customers in waiting area,In production,Neither,1/1/2023,7/1/2023,NA,NA,Developed with both contracting and in-house resources.,NA,No,NA,None,Yes,NASA's data stewardship best practices are in effect under guidance from the Chief Data Officer.,No,No,No,Blank/Not Provided,None,No,N/A,,Yes,Yes,NA,Yes,AWARE,Less than 1 Year,No,,No,,,,"This project re-used prior work in identifying vehicles, objects or animals in a NASA environment",Yes,,,,,,,,,,,,,,,,,, Onboard Planner for Mars2020 Rover (Perseverance),National Aeronautics and Space Administration,NASA,Jet Propulsion Laboratory,Other,None,None of the above.,"The M2020 onboard scheduler incrementally constructs a feasible schedule by iterating through activities in priority-first order. When considering each activity it computes the valid time intervals for placement, taking into account preheating, maintenance heating, and wake/sleep of the rover as required. After an activity is placed (other than a preheat/maintenance or wake/sleep), the activity is never reconsidered by the scheduler for deletion or moving. Therefore the scheduler can be considered non backtracking, and only searches in the sense that it computes valid timeline intervals for legal activity placement. Meta Search: Because the onboard scheduler will be invoked many times in a given sol (Martian Day) with a range of possible contexts (due to execution variations), its non backtracking nature leaves its vulnerable to brittleness. In order to mitigate this potential brittleness, the Copilot systems perform a monte carlo based stochastic analysis to set meta parameters of the scheduler - primarily activity priority but also potentially preferred time and temporal constraints. Also: Research, experiments, and engineering to empower future rovers with onboard autonomy; planning, scheduling & execution; path planning; onboard science; image processing; terrain classification; fault diagnosis; and location estimation. This is a multi-faceted effort and includes experimentation and demonstrations on-site at JPL's simulated mars navigation yard.",Planning and scheduling recommendations,In mission,Neither,1/1/2016,1/1/2018,7/1/2020,NA,Developed with both contracting and in-house resources.,NA,No,NA,None,No,NASA's data stewardship best practices are in effect under guidance from the Chief Data Officer.,No,No,Yes,Blank/Not Provided,"As a Mission-embedded system on Mars Rover, ATO, public dissemination, SAOP, HISP, etc. do not apply",Yes,N/A,,Yes,Yes,NA,No. Engineering Review and Resease instead.,Multiple approved space-based systems,Less than 6 months,No,,No,,,,This work built-upon prior rovers,Yes,,,,,,,,,,,,,,,,,, "SensorWeb: Volcano, Flood, Wildfire, and others.",National Aeronautics and Space Administration,NASA,Jet Propulsion Laboratory,Statistical Methods,None,None of the above.,"The Sensor Web Project uses a network of sensors linked by software and the internet to an autonomous satellite observation response capability. This system of systems is designed with a flexible, modular, architecture to facilitate expansion in sensors, customization of trigger conditions, and customization of responses. This system has been used to implement a global surveillance program to study volcanos. We have also run sensorweb tests to study flooding, cryosphere events, and atmospheric phenomena. Specifically, in our application, we use low resolution, high coverage sensors to trigger observations by high resolution instruments. Note that there are many other rationales to network sensors into a sensorweb. For example automated response might enable observation using complementary instruments such as imaging radar, infra-red, visible, etc. Or automated response might be used to apply more assets to increase the frequency of observation to improve the temporal resolution of available data. Our sensorweb project is being used to monitor the Earth's 50 most active volcanos. We have also run sensorweb experiments to monitor flooding, wildfires, and cryospheric events (snowfall and melt, lake freezing and thawing, sea ice formation and breakup.)","Identification and labelling of terrain, climate and weather features",In mission,Neither,1/1/2003,1/1/2003,1/1/2003,NA,Developed with both contracting and in-house resources.,NA,No,NA,None,No,NASA's data stewardship best practices are in effect under guidance from the Chief Data Officer.,No,No,Yes,Blank/Not Provided,Remote sensing data from many scientific instruments,Yes,N/A,,Yes,Yes,NA,No. Engineering Review and Resease instead.,Multiple approved space-based systems,Less than 6 months,No,,No,,,,This capability has been re-used among many missions,Yes,,,,,,,,,,,,,,,,,, "Global, Seasonal Mars Frost Maps",National Aeronautics and Space Administration,NASA,Jet Propulsion Laboratory,Deep Learning,None,None of the above.,Global frost Martian maps dervied from five remote sensing datasets and processed with tools like CNNs and other data science techniques. Publicly available on JMARS,Frost maps,In production,Neither,5/1/2021,10/1/2021,11/1/2024,NA,Developed with both contracting and in-house resources.,NA,No,NA,None,Yes,NASA's data stewardship best practices are in effect under guidance from the Chief Data Officer.,No,No,Yes,Blank/Not Provided,"HiRISE, CTX, THEMIS, MCS, CRISM",Yes,N/A,,Yes,No – agency does not have access to source code.,NA,No. Engineering Review and Resease instead.,Multiple approved space-based systems,Less than 6 months,No,,Yes,,,,This capability has been re-used among many missions,Yes,,,,,,,,,,,,,,,,,, Purchase Card Management System (PCMS),National Aeronautics and Space Administration,NASA,Jet Propulsion Laboratory,NLP,None,None of the above.,Purchase card application uses ML model to suggest if a purchase may be a taggable asset or a chemical.,Metadata tagging recommendation,In production,Neither,11/13/2018,11/13/2018,11/15/2018,NA,Developed with both contracting and in-house resources.,NA,No,NA,None,No,NASA's data stewardship best practices are in effect under guidance from the Chief Data Officer.,No,No,No,Blank/Not Provided,The training data set relies on historical records related to the description of purchased items along with the corresponding Asset-Tag/Chemical flag to determine the suggested item categorization,Yes,N/A,,Yes,Yes,NA,No. Engineering Review and Resease instead.,,Less than 6 months,No,,No,,,,We may share this technique with other relevant use cases,Yes,,,,,,,,,,,,,,,,,, New Technology and Software Reporting (NTR),National Aeronautics and Space Administration,NASA,Jet Propulsion Laboratory,NLP,None,None of the above.,"NTR application uses ML model to suggest a Technology Category (e.g. Aerospace, Robotics) for the new technology being reported",Category classification,In production,Neither,5/9/2024,7/2/2024,8/22/2024,NA,Developed with both contracting and in-house resources.,NA,No,NA,None,No,NASA's data stewardship best practices are in effect under guidance from the Chief Data Officer.,No,No,No,Blank/Not Provided,The training data set relies on historical records related to the description of the New Technolgy Reported along with the corresponding Technology Category field to determine the suggested categorization,Yes,N/A,,Yes,Yes,NA,No. Engineering Review and Resease instead.,,Less than 6 months,No,,No,,,,We may share this technique with other relevant use cases,Yes,,,,,,,,,,,,,,,,,, "ServiceNow GenAI ""Now Assist""",National Science Foundation,NSF,OCIO,Mission-Enabling,,Collaborating in real-time using AI-assisted tools in word processors.,"ServiceNow GenAI is a FedRAMP High certified system to accelerate digital transformation using Now Assist. Automatically generate content such as responses, work notes, and knowledgebase articles related to customer service experience. Increase productivity and speed to delivery with intelligent recommendations. Resolve issues swiftly with chatbots that understand human language.",Increase productivity and speed to delivery with intelligent recommendations. Resolve issues swiftly with chatbots that understand human language.,Acquisition and/or Development,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Budget Area Topic Classification Tool,National Science Foundation,NSF,CISE/OAD,Mission-Enabling,,Summarizing the key points of a lengthy report using AI.,"The Budget Area Topic Classification Tool uses a pre-trained BERT model ('bert-base-uncased') to classify proposals into 1 of 8 budget areas: Advanced Manufacturing, AI, Advanced Wireless, Clean Energy, Microelectronics & Semiconductors, Quantum, or Trustworthy AI. This approach is semi-supervised in nature as the training data is comprised of pre-tagged CISE proposals from prior years. Upon training the model to learn the multi-label structure of the underlying data, the model's accuracy and discriminatory power was tested using a hold-out sample (i.e. 80/20 split). ",The model outputs a numerical vector of probabilities representing the likelihood that a given proposal belongs to 1 or more categories. A cut-off value can then be set for determining final classification for a given proposal.,Acquisition and/or Development,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, AI voice over,National Science Foundation,NSF,OLPA,Mission-Enabling,,Transcribing and summarizing a recorded meeting or interview using AI.,"Commercial-off-the-shelf (COTS) - The AI system was purchased pre-built from a third-party, Wellsaid has been used to generate placeholder voice over, which will be replaced later by real voice over, so it doesnt cause much AI policy concerns. ",allowing the agency to skip design and development by federal or contract personnel.,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Adobe Photoshop,National Science Foundation,NSF,OLPA,Mission-Enabling,,Creating visually appealing presentations using AI-driven design suggestions.,OLPA's Creative Services team will use Photoshop's AI-powered tools ,"to refine edits of photos and graphics, particularly the retouching of portraits.",Implementation and Assessment,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Resubmit Checker ,National Science Foundation,NSF,EDU/DRL,Mission-Enabling,,Searching for information using AI.,"The resubmit checker takes a list of proposal IDs and returns a list of similar proposals submitted to NSF in the past, scored by their similarity. ","This is useful for identifying resubmitted proposals for several purposes: evaluating whether to return without review for lack of revision, ensuring continuity of merit review, and the analysis of broader resubmit trends.",Implementation and Assessment,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, AWS Assisted Software Development,National Science Foundation,NSF,OCIO,Mission-Enabling,Developer Tools,None of the above.,"Within software development, AI-driven tools, such as AWS CodeWhisperer, Amazon Q and Bedrock, and integration with development workflow, promise to elevate developer productivity and refine code quality. .","The objective of this pilot is to explore potential integration of these AWS services with DIS Developers IDEs, Jenkins automation, Bitbucket, laying the groundwork for Secure AI Assisted Development at NSF",Acquisition and/or Development,Neither,7/15/1905,7/15/1905,,,,,No,,,No,This is a COTS solution and we are using industry best practices and OEM provided solution ,No,Yes,No,,Training data will be derived from both AWS pre trained models as well as Agency generated in the case of Bedrock tuning to NSF use cases,"Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",N/A,,No,No – agency does not have access to source code.,,No,,,No,,,,,,,,,,,,,,,,,,,,,,,,, Mine Potential Reviewer's Area of Expertise,National Science Foundation,NSF,TIP-ITE-ENGINES,Mission-Enabling,,Searching for information using AI.,Airtable is a low code platform that incorporates Open AI. The Open AI services are purchased by and the data is kept private to Airtable. TIP has been approved to use Airtable for several usecases and Airtable has been certified as secure by the NSF OCIO Security Team. , to use Airtable to trial Airtable's AI's ability to match potential reviewer expertise to the reviewer.,Acquisition and/or Development,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Finding the State,National Science Foundation,NSF,TIP-ITE-ENGINES,Mission-Enabling,,Searching for information using AI.,"TIP is authorized to use Airtable for project management including reviewer volunteers and recruitment. Airtable is a no-code platform that includes a built-in database. We do not store any PII in our database except for "public PII" such as name, position, work email address, areas of professional expertise, etc. ",1. Airtable AI is prompted to look at the potential reviewers Linked In profile and identify or update the potential reviewers expertise in its database based on recent updates to professional experience in LinkedIn 2 Search existing potential Submitters database for state associated with submitters institution ,Implementation and Assessment,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Enhancing TIP Awards: Efficient Tagging for the 10 Key Technology Areas (KTA),National Science Foundation,NSF,TIP/OAD,Mission-Enabling,,Searching for information using AI.,"Per the CHIPS and Science Act mandate, TIP needs to know our investment portfolio based on the 10 KTA.",Enhancing TIP Awards: Efficient Tagging for the 10 Key Technology Areas (KTA),Acquisition and/or Development,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Data Access Alternatives: Artificial Intelligence Supported Interfaces,National Science Foundation,NSF,SBE/NCSES,Mission-Enabling,,Searching for information using AI.,The key objective of this effort is to create and test machine-learning-backed or “artificial intelligence” (AI)-backed user experiences with federal statistical data. This experience shall improve on the current state of user interactions based on obtaining answers to questions via search engines or emailing federal staff or contractors. ,This project seeks to develop and pilot an AI chat bot (or the like) that answers users text queries submitted via an interface. Answers will be obtained from public statistical data of federal statistical agencies in this project.,Initiated,Neither,7/16/1905,,,,Developed with both contracting and in-house resources.,,No,,,,,,Yes,Other,Currently we are piloting if this is possible. This chat bot will answer questions. The chat bot will return statistical public data. Users will always be able to contact the agency via email or phone.,,,,,Yes,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, FOIA processing,National Science Foundation,NSF,OD/OGC,Mission-Enabling,,Searching for information using AI.,"No tool has been selected yet, exploring tools tht may help with sorting requests",,Initiated,Neither,7/16/1905,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Alt text Generator for Images in NSF's Media Hub,National Science Foundation,NSF,OD/OLPA,Mission-Enabling,,Creating visual representations of data sets for reports and presentations using AI.,"Exploring using a ChatGPT integration with NSF’s Digital Asset Management system (DAM), for the images we host there. ", to generate alt text for images we'll be using in digital communication products (such as nsf.gov) to ensure they’re accessible to individuals using assistive technology.,Initiated,Neither,7/16/1905,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Exploring the use of BERTopic Modeling for a portfolio analysis,National Science Foundation,NSF,OIA/EAC,Mission-Enabling,,Searching for information using AI.,Exploring whether the application of BERTopic modeling (or a similar NLP method) is applicable to a portfolio analysis. ,xploring whether it is applicable to portfolio analysis,Initiated,Neither,7/16/1905,,,,Developed with both contracting and in-house resources.,,No,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Open-source topic identification for the Public Engagement with Science Initiative data analysis.,National Science Foundation,NSF,OD/OIA,Mission-Enabling,,Searching for information using AI.,Exploring the use of unsupervised NLP to identify topical themes within responses submitted to the NSF public engagement questionnaire. The AI system will include multiple open-source techniques used together to achieve this goal.,"To develop a cluster model to be applied to all responses. Where feasible, the RTI STEM education experts/analysts will manually map the research topics to NSF research programs.",Initiated,Neither,7/16/1905,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Hand-written text extraction using Textract for the Public Engagement with Science Initiative,National Science Foundation,NSF,OD/OIA,Mission-Enabling,,Digitizing text from scanned documents or smart forms for archival purposes using AI.,"We propose using a Commercial Off the Shelf AWS product, Textract, as a character recognition tool Handwritten text will be an open format question for science and engineering and some optional check boxes for demographic and other data questions.", to convert handwritten text into machine-encoded text for a postcard.,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Ask NSF Initial Pilot,National Science Foundation,NSF,OCIO/DDAI,Mission-Enabling,,Searching for information using AI.,Ask NSF initial pilot is intended to test the hypothesis that a chatbot can be rapidly created using an AI Platform.,Answer general questions from existing documents,Initiated,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, USA Class,Office of Personnel Management,OPM,HRS/HRSES/Federal Classification Center,Mission-Enabling,,,"The purpose of this project is to build an application to help managers and HR specialists (classifiers) improve the quality and speed the development of position descriptions (PDs). Developing a PD can be a lengthy process that requires significant back-and-forth between managers and classifiers. The length of time required to develop a PD is often cited as a major pain point and bottleneck in federal hiring, workforce planning, organizational design, and performance management processes. To address the challenge today, managers will simply copy whole PDs or significant sections of PDs which weakens the accuracy of the new PD. Agencies have also turned to the use of “standard” PDs which tend to be too generic to be effective for use in recruitment and performance. This project seeks to solve the “blank-page-syndrome” by suggesting draft major duties, factor evaluation statements, and other elements of the PD for managers and classifiers to use to develop PDs that meet classification standards. This project is expected to significantly decrease the time required to develop PDs, leading to a decrease in time to hire and other related HR processes. A decrease in the time to hire means agencies can be more competitive in attracting top talent to their organizations, enhancing their ability to meet their missions. A decrease in the time required to develop position descriptions is also expected to lead to cost savings for agencies – freeing up manager and HR specialist time for other tasks. Additionally, the streamlined generation of PDs will enable agencies to move away from “standard” PD processes which will allow for better organizational design and workforce planning outcomes. The project is expected to also lead to higher-quality PDs that adhere to classification standards. This will improve position and performance management and talent acquisition in agencies, ensuring organizations are structured and staffed appropriately to meet their missions. ","USA Class will provide draft PD content generated through large language models (LLMs) and informed by a PD library curated by OPM experts and user responses to prompt questions about the position’s purpose, regular and recurring work, and knowledge required. The questions are intended to be straightforward and easily answered by the manager in a few minutes. The application will then send this information in a prompt to the LLM and generates suggested major duty titles and work tasks. The user (manager or classifier) can edit the draft content suggested by the system and makes the decision whether to include it in the PD or not. Future iterations will include a certification by a classifier and others that they accept/approve the final content of the full PD.",Acquisition and/or Development,"Rights-Impacting ",3/1/2024,3/1/2024,,,Developed with both contracting and in-house resources.,24322624F0027,No,,,,,,,Yes,,"The AI system uses a RAG (Retrieval-Augmented-Generation) architecture including GPT-4o (accessed through OPM’s Azure Open AI environment) that leverages manager-provided responses to a few brief questions as well as a library of curated and validated federal position descriptions. The Position Description library consists of approximately 2,800 PDs developed by OPM/HRS’s Federal Classification Center for internal and external customer agencies.","Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",,,,,,Yes,USAData Analytics,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,Yes,"Benchmark evaluation: Testing of the AI model has involved the use of benchmarks (either publicly available or internally created) to estimate performance in real-world settings, but has not been tested in an operational environment. ","We are aware that AI models can perpetuate existing biases present in the training data, leading to unfair or discriminatory classification outcomes. It could be a risk to user adoption that Federal managers and HR professionals do not understand how USA Class uses generative AI and whether or not it is safe and effective for them to use. USA Class could produce incomplete, inaccurate, or inconsistent suggestions due to limitations in training data or other gaps. USA Class users could rely too much on the suggested position description content and not review and edit or adjust their prompts enough to get as precise results as are possible leading to overly generic or incomplete PDs. ",Yes – by an agency AI oversight board not directly involved in the system’s development,No monitoring protocols have been established: Necessary infrastructure needed to perform monitoring of AI systems in production is not available and/or a plan to monitor models has not yet been established.,No - Some individual decisions or actions require direct human oversight.,,Online - in the terms or instructions for the service.,,No,These activities are expected as the current prototype matures in the pilot process.,None of the above,,Yes,,Yes, Copilot for Microsoft 365,Office of Personnel Management,OPM,OCIO/CTO,Mission-Enabling,,Searching for information using AI.,"Copilot for Microsoft 365 is an AI-powered addition to the Microsoft 365 suite that helps users be more productive and creative by working with Microsoft 365 apps and services. It uses large language models (LLMs) and integrates with a user's data and Microsoft Graph. Here are some of the ways that Copilot can assist OPM users in Microsoft 365: • Summarize • Draft • Analyze data • Create, edit, and share • Automate tasks • Suggest ideas • Personalize","The expected output for Copilot for Microsoft 365 are: 1) Summarizing emails, meetings, calls, and chats. 2) Drafting emails and provide coaching on drafts. 3) Analyzing data and identifying insights in Excel. The AI would not make decisions for the users. Instead, it will act as a productivity tool to summarize content and create first drafts for the user to edit. It is the OPM user’s responsibility to validate all content and edit as necessary.",Acquisition and/or Development,"Rights-Impacting ",,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Insight,Social Security Administration,SSA,"Office of Analytics, Review, and Oversight",Government Services (includes Benefits and Service Delivery),,None of the above.,"This AI use case is decision support software used by hearings and appeals-level Disability Program adjudicators to help maximize the quality, speed, and consistency of their decision making. The use case analyzes the free text of disability decisions and other case data to offer adjudicators real-time alerts on potential quality issues and case-specific reference information within a web application. The use case increases decision quality and reduces human error.","Insight provides 43 Quality flags, when raised, indicating various inconsistencies, omissions, or Policy non-compliant conditions that exist in the disability decision. ",Operation and Maintenance,Neither,7/24/2015,10/1/2016,10/1/2017,,Developed with both contracting and in-house resources.,ITSSC,Yes,Social Security Administration – SSA ,Obtaining audit disability benefits,No,,Yes,Yes,Yes,,Manually curated data sets based on random sampling methodology of hearing and appeals case processing management information,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ","Age, Income, Employment Status",,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Insight,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, Intelligent Medical Language Analysis Generation (IMAGEN),Social Security Administration,SSA,Office of the Chief Information Officer,Government Services (includes Benefits and Service Delivery),,Searching for information using AI.,"This AI use case helps employees visualize, search, and more easily identify relevant clinical content in medical records. The use case improves efficiency and accuracy of disability determinations/decisions.","Document annotations on medical records to aid in navigation such as identifying patient encounters, tests, dates, and other relevant information. Possible impairment and SSA Medical Listing Codes are also identified. ",Operation and Maintenance,"Rights-Impacting ",1/1/2019,1/1/2019,8/20/2021,,Developed with both contracting and in-house resources.,Agency-wide IT Support Services Contract. (ITSSC),Yes,Social Security Administration – SSA ,Obtaining audit disability benefits,No,,Yes,Yes,Yes,,Historically adjudicated disability claims and outcomes. Manually annotated documents,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",Other,"Models do not use structured information regarding demographic variables. However, because medical records are the input for the model, demographic variables may be included as unstructured data. ",Yes,"Yes – agency has access to source code, but it is not public.",,Yes,IMAGEN,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Duplicate Identification Process,Social Security Administration,SSA,Office of the Chief Information Officer,Government Services (includes Benefits and Service Delivery),,None of the above.,"This AI use case identifies, flags, and marks duplicates, reducing the amount of time spent to review cases for hearings.","This AI use case identifies, flags, and marks duplicates, reducing the amount of time spent to review cases for hearings.",Operation and Maintenance,Neither,8/20/2018,10/1/2018,9/1/2019,,Developed with contracting resources.,"28321324D00060010 ITSSC - Accenture",Yes,Social Security Administration – SSA ,Obtaining audit disability benefits,No,,Yes,Yes,No,,No Agency data used for training,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,No,No – agency does not have access to source code.,,Yes,DIP,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Handwriting Recognition of Forms; Intelligent Document Processing Platform - Forms Classification and Data Extraction ,Social Security Administration,SSA,Office of the Chief Information Officer,Government Services (includes Benefits and Service Delivery),,None of the above.,"This AI use case classify, identify and extract structured, semi-structured forms containing printed and handwritten fields. The use case increases automation, reduces clerical errors and reduces customer wait times.",The output of extracted data fields on the form is represented in JSON format that can be available to downstream systems using API and through other integration mechanism such as queues.,Operation and Maintenance,Neither,3/1/2020,11/1/2020,1/2/2021,,Developed with contracting resources.,"28321324D00060010 ITSSC - Accenture",Yes,Social Security Administration – SSA ,Obtaining audit disability benefits,No,,Yes,Yes,No,,No Agency data used for training,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,No,No – agency does not have access to source code.,,Yes,"Form Data Extraction ",Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Modernized Development Worksheet (MDW),Social Security Administration,SSA,"Office of Analytics, Review, and Oversight",Government Services (includes Benefits and Service Delivery),,None of the above.,This AI use case reviews data during case development so it can be better categorized. The use case reduces processing time and customer wait times.,Labels and categorizes new or updated MDW requests,Operation and Maintenance,Neither,7/11/1905,Current model: 2019-2023,12/1/2023,N/A,Developed in-house.,,Yes,Social Security Administration – SSA ,Obtaining audit disability benefits,No,,Yes,No,Yes,,Model trained from case development worksheets prepared by SSA employees.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,No,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Anomalous iClaim Predictive Model,Social Security Administration,SSA,"Office of Analytics, Review, and Oversight",Government Services (includes Benefits and Service Delivery),,None of the above.,This AI use case identifies high-risk iClaims. These claims are then sent to Operations for further review before additional action is taken to adjudicate the claims.,Risk scores for recently submitted iClaims.,Operation and Maintenance,"Rights-Impacting ",10/1/2015,10/1/2015,3/20/2016,,Developed in-house.,,Yes,Social Security Administration – SSA ,Filing for Social Security retirement benefits,No,,Yes,No,Yes,,Disclosing this information may impede the agency's ability to effectively combat fraud.,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",Disclosing this information may impede the agency's ability to effectively combat fraud.,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Anti-Fraud Infrastructure,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Pre-Effectuation Review / Targeted Denial Review Models,Social Security Administration,SSA,"Office of Analytics, Review, and Oversight",Government Services (includes Benefits and Service Delivery),,None of the above.,This AI use case identifies high likelihood of error in certain claims and refers them for review. This use case reduces incorrect benefits payment.,Probability scores via an in-line mainframe process,Operation and Maintenance,"Rights-Impacting ",1/1/1996,Current model: 2020-2024,12/1/2024,N/A,Developed in-house.,,Yes,Social Security Administration – SSA ,Obtaining audit disability benefits,No,,Yes,No,Yes,,Models used data from disability program data sets.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",Age,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,No,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Representative Payee Misuse Model,Social Security Administration,SSA,"Office of Analytics, Review, and Oversight",Government Services (includes Benefits and Service Delivery),,None of the above.,This AI use case identifies possible representative payee fraud and flags for review. This use case reduces representative payee fraud.,The system outputs numerical scores based on likelihood of representative payee misuse.,Operation and Maintenance,"Rights-Impacting ",1/1/2009,1/1/2009,9/30/2023,N/A,Developed in-house.,,Yes,Social Security Administration – SSA ,Obtaining audit disability benefits,No,,Yes,No,Yes,,Models used data from earnings and beneficiary data.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.","Age, Employment Status, Ability Status",,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,No,,Yes,,Other,We have not needed to allocate resources.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Continuing Disability Review Model,Social Security Administration,SSA,"Office of Analytics, Review, and Oversight",Government Services (includes Benefits and Service Delivery),,None of the above.,This AI use case identifies disability cases with the greatest likelihood of medical improvement and flag them for a continuing disability review to improve the accuracy of CDRs.,"The system outputs numerical scores for each disability case: the score represents the likelihood of medical improvements. The probabilities are binned according to low, medium, or high. ",Operation and Maintenance,"Rights-Impacting ",1/1/1993,1/1/1993,9/30/2018,N/A,Developed in-house.,,Yes,Social Security Administration – SSA ,Obtaining audit disability benefits,No,,Yes,No,Yes,,Model used data from disability program data sets,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.","Age, Ability Status",,Yes,"Yes – agency has access to source code, but it is not public.",,No,,,Yes,,Yes,,Other,We have not needed to provision resources.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Social Security Income Redetermination Model,Social Security Administration,SSA,"Office of Analytics, Review, and Oversight",Government Services (includes Benefits and Service Delivery),,None of the above.,This AI use case identifies SSI overpayment cases that have highest expected overpayments for review. This use case contains excess overpayments by prioritizing technician review,"The system outputs numerical scores for individuals, comprised of likelihood and amount of overpayment. ",Operation and Maintenance,"Rights-Impacting ",1/1/1995,1/1/1995,9/9/2024,N/A,Developed in-house.,,Yes,Social Security Administration – SSA ,Obtaining audit disability benefits,No,,Yes,No,Yes,,Model used data from disability program data sets,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",Wages (Employment Status),,Yes,"Yes – agency has access to source code, but it is not public.",,No,,,Yes,,Yes,,Other,We have not needed to provision resources.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Proactive Triage and Analysis of Hearings (PATH),Social Security Administration,SSA,"Office of Analytics, Review, and Oversight",Government Services (includes Benefits and Service Delivery),,None of the above.,This AI use case flags high likelihood favorable claims and refers them to human adjudicators for further review to determine eligibility for on-the-record hearing decisions.,The system outputs sorted case listings based on likelihood of favorable hearing decision.,Operation and Maintenance,"Rights-Impacting ",1/1/2018,1/1/2018,11/1/2024,N/A,Developed in-house.,,Yes,Social Security Administration – SSA ,Obtaining audit disability benefits,No,,Yes,No,Yes,,Model used data from disability program data sets,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",Age,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,No,,Yes,,Other,We have not needed to provision resources.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Quick Disability Determinations Model,Social Security Administration,SSA,Office of Retirement and Disability Policy,Government Services (includes Benefits and Service Delivery),,None of the above.,Used to screen initial applications to identify cases where a favorable disability determination is highly likely and medical evidence is readily available to prioritize this workload and expedite case processing. ," System output consists of probability model scores for each separate scoring service, Scoring Service for Allowance and Scoring Service for Processing Time, and the overall QDD Score. ",Operation and Maintenance,"Rights-Impacting ",9/25/2003,9/15/2004,2/1/2008,,Developed with contracting resources.,SS0004400360001,Yes,Social Security Administration – SSA ,Obtaining audit disability benefits,No,,Yes,Yes,Yes,,Model used data from disability program data sets,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ","Sex, gender, & age",,Yes,No – agency does not have access to source code.,,Yes,eDIB (electronic disability),6-12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Mobile Wage Reporting (MOBWR) ,Social Security Administration,SSA,Office of the Chief Information Officer,Government Services (includes Benefits and Service Delivery),,Inputting large amounts of data from paper forms into a digital system using AI.,Uses AI to extract text/data from scanned images/documents representing pay stubs or payroll information to enable faster processing.,Gross pay and pay date from the paystub image,Operation and Maintenance,Neither,11/4/2020,3/1/2021,5/14/2022,,Developed with contracting resources.,"6801, 10128, 10191, 10325",Yes,Social Security Administration – SSA ,Obtaining audit disability benefits,No,,Yes,Yes,No,,No Agency data used for training,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,No,No – agency does not have access to source code.,,Yes,Supplemental Security Income Record Maintenance System (SSIRMS),Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, SSA 800# Public Question & Answer Bot,Social Security Administration,SSA,Office of Transformation,Government Services (includes Benefits and Service Delivery),,Searching for information using AI.,"The bot answers caller’s questions using approved FAQs found on SSA.GOV. Support is provided in both English and Spanish. Calls answered solely by the bot reduce the wait for other callers needing agent service.",Returns the answer snippet content for the applicable FAQ topic that should contain the same language as if the askers were to navigate to the FAQs webpage themselves on ssa.gov.,Operation and Maintenance,Neither,2/13/2023,4/3/2023,6/5/2023,,Developed with contracting resources.,,Yes,Social Security Administration - SSA,Filing for Social Security retirement benefits,Yes,N/A - Information Correction actions are not supported.,No,No,Yes,,The ssa.gov FAQs webpage content is exported to be sourced by the FAQ Question & Answer bot.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,Yes,No – agency does not have access to source code.,,Yes,Enterprise Contact Center,Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Generative AI for Analytic Insights,Social Security Administration,SSA,"Office of Analytics, Review, and Oversight",Government Services (includes Benefits and Service Delivery),,None of the above.,"Generative AI model to analyze SSA's National 800 Number transcripts to provide insights into caller intent, call resolution, and call topics. These insights could highlight pain points in SSA’s policies and processes and focus efforts to improve customer service, deflect callers whose business cannot be resolved using the 800 Number, and ultimately decrease wait times.",Varies by use case,Operation and Maintenance,Neither,10/1/2022,10/1/2022,9/30/2023,N/A,Developed with contracting resources.,,Yes,Social Security Administration – SSA ,Obtaining audit disability benefits,No,,Yes,No,No,,No Agency data used for training,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,No,No – agency does not have access to source code.,,Yes,*** Provisional ATO in progress,6-12 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, PolicyNet Plain Language Search Engine,Social Security Administration,SSA,Office of the Chief Information Officer,Government Services (includes Benefits and Service Delivery),,Searching for information using AI.,"Search will employ learning to rank functionality based on user entered thumbs up/thumbs down data (data capture began in FY24 - learning to rank will be implemented in FY25). Users across all internal SSA components and offices may use the system to find relevant information on SSA policies, procedures or instructions. Learning to rank will help users find relevant results faster, which supports better customer service. ","Based on the user thumbs up/thumbs down selections, search results will be ranked higher (or lower) within the results output. ",Operation and Maintenance,Neither,3/1/2024,8/19/2024,9/21/2024,N/A,Developed with contracting resources.,ITSSC,Yes,Social Security Administration - SSA,Filing for Social Security retirement benefits,No,,No,Yes,No,,Model used disability program regulations and user provided data,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,Yes,Yes – source code is publicly available.,https://solr.apache.org/,Yes,21st Century PolicyNet ,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Speech to Text Video Transcription,Social Security Administration,SSA,Office of the Chief Information Officer,Mission-Enabling,,Transcribing and summarizing a recorded meeting or interview using AI.,Generate transcript from recorded audio of disability hearings,Transcript,Operation and Maintenance,Neither,10/1/2023,10/1/2023,5/6/2024,,Developed with contracting resources.,Microsoft/Accenture,Yes,Social Security Administration - SSA,Filing for Social Security retirement benefits,No,,Yes,No,No,,No Agency data used for training,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,No,No – agency does not have access to source code.,,Yes,Hearing Recording and Transcription,,Yes,,Other,Model is SaaS ,Other,N/A,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Customer Insight Tool,Social Security Administration,SSA,Office of Transformation,Government Services (includes Benefits and Service Delivery),,None of the above.,"Survey analysis tool that evaluates sentiment and categorization. Current capabilities include: 1. Machine Translation - using algorithms to automatically translate text from one language to another 2. Sentiment Analysis - classifies text into negative, positive and neutral sentiment 3. Categorization - system-generated groupings of conceptually-related words and phrases based on phrase-level feedback 4. Suggested Actions - uncovers explicit suggestions from feedback text 5. Customer Effort - aimed to recover explicit mentions of effort in feedback text","Survey analysis tool that evaluates sentiment and categorization. Current capabilities include: 1. Machine Translation - using algorithms to automatically translate text from one language to another 2. Sentiment Analysis - classifies text into negative, positive and neutral sentiment 3. Categorization - system-generated groupings of conceptually-related words and phrases based on phrase-level feedback 4. Suggested Actions - uncovers explicit suggestions from feedback text 5. Customer Effort - aimed to recover explicit mentions of effort in feedback text",Operation and Maintenance,Neither,8/1/2021,9/26/2020,8/1/2021,,Developed with contracting resources.,Inter-Agency Agreement (IAA-21-0017),Yes,Social Security Administration - SSA,Obtaining audit disability benefits,No,,Yes,No,No,,No Agency data used for training,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,No,No – agency does not have access to source code.,,Yes,Customer Feedback Tool (CXFT),Less than 6 months,Yes,,Other,Third Party Vendor,Other,Third Party Vendor ,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Benefit and Earnings Public Use File,Social Security Administration,SSA,Office of Retirement and Disability Policy,Government Services (includes Benefits and Service Delivery),,None of the above.,Generate high fidelity synthetic benefits and earnings data for external researches to support policy research and decision making while protecting privacy. ,Synthetic microdata file aimed for public use by policy researchers,Operation and Maintenance,Neither,9/1/2023,,,,Developed in-house.,,No,,,Yes,Comprehensive Data Quality Review,Yes,Yes,Yes,,Model used data from disability program data sets,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ","Sex, Age, Marital Status, Earnings",,Yes,"Yes – agency has access to source code, but it is not public.",,No,,,No,0,No,,Other,None requested,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, Therapy Chatbot - Text-Based Mental Health Support for SSA Employees,Social Security Administration,SSA,Office of Human Resources,Health & Medical,,None of the above.,This AI use case is a therapy chatbot that employees can access on their own personal devices.,Synthetic text output that serves discussion with users,Operation and Maintenance,Safety-Impacting,,7/1/2023,9/1/2023,,Developed with contracting resources.,,No,,,No,NA,Yes,,No,,No Agency data used for training,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,No,No – agency does not have access to source code.,,No,,,,,Other,Third Party Vendor,Other,Third Party Vendor ,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Agency Support Companion (ASC) chatbot,Social Security Administration,SSA,Office of the Chief Information Officer,Mission-Enabling,,Improving the quality of written communications using AI tools.,"Generative AI model to create content, summarization and research capabilities",Chatbot responses to users as well as management information reports for executives,Acquisition and/or Development,Neither,7/15/2024,8/14/2024,,,Developed with contracting resources.,ITSSC,No,,,No,,No,No,No,,No Agency data used for training,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,No,Agency Support Companion,,Yes,,No,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Intelligent generation of modernized code.,Social Security Administration,SSA,Office of the Chief Information Officer,Government Services (includes Benefits and Service Delivery),,None of the above.,"The module uses AI techniques such as cluster analysis, pattern matching, templates, and programming language models to generate modernized code in a new target language. These AI techniques enable it to perform efficient data analysis, processing, and intelligent generation of new code. AI powered approach provides structured standardized code that’s highly maintainable. ",Transformed legacy code into modernized language,Operation and Maintenance,Neither,10/1/2024,7/1/2024,11/1/2014,,Developed with contracting resources.,ITSSC,No,,,No,NA,No,No,No,,No Agency data used for training,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,No,No – agency does not have access to source code.,,No,ISAT,Less than 6 months,Yes,,No,Third Party Vendor,Other,Third Party Vendor,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Data Governance Product (DGP) - Data cataloging and predictive analysis of data assets. ,Social Security Administration,SSA,Office of the Chief Information Officer,Government Services (includes Benefits and Service Delivery),,None of the above.,"Vendor built in AI engine for data cataloging to automates the discovery, classification, and curation of data assets. And also provides predictive insights for data asset analytics. Uses Natural Language Processing (NLP) enables non-technical stakeholders to contribute to rule creation and facilitates the conversion of plain text into regular expression. ",Efficient data catalog search results. Plain English conversation with the users to create technical regular expression rule creation,Operation and Maintenance,Neither,5/1/2020,7/11/2022,10/1/2024,,Developed with contracting resources.,contract number GS-35F-267DA,No,,,No,NA,Yes,Yes,Yes,,Model used data from internal sources for performance and evaluation purposes,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Data Governance Product (DGP),Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Product Service Code Automation ML Model,Department of State,STATE,A/GO,Mission-Enabling (internal agency support),N/A,N/A,"Developed a proof-of-concept machine learning model to scan unstructured, user entered procurement data such as Requisition Title and Line Descriptions to automatically detect the commodity and service types being purchased for enhanced procurement categorization. This project is retired.",N/A it was a proof of concept that has been cancelled.,Retired,Neither,,,,9/1/2021,N/A,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A Tailored Integration Logistics Management System (ILMS) User Analytics,Department of State,STATE,A/GO,Mission-Enabling (internal agency support),N/A,N/A,Piloted the use of ILMS transactional data and planned transactions to develop tailored user experiences and analytics to meet the specifics needs of the user at that moment. This allowed for the extraction of more meaningful information about users by mining real system actions and clicks. This project is retired.,N/A this was a pilot project that is not in production and was cancelled. ,Retired,Neither,6/1/2022,6/1/2022,6/1/2022,6/14/2022,N/A,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A Property and Procurement Analytics,Department of State,STATE,A/GO,Mission-Enabling (internal agency support),N/A,N/A,Plan to expand current analytics within the Integrated Logistics Management System (ILMS) through development of a machine learning model for detecting patterns of potential anomalous activity in property and procurement.,Plan to have data on potential anomalous activity.,Planned,Neither,6/1/2022,6/1/2022,N/A,N/A,N/A,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A AI input in Translation,Department of State,STATE,A/PRI/LS,Mission-Enabeling ,N/A,None of the above.,Leveraging machine-based tools to streamline workflows in translation work. Machine output is carefully post-edited by professional human translators to ensure highest quality product. ,Very rough suggested language that is carefully reviewed and edited by human professional translators. ,Implementation and Assessment,Neither,1/19/2024,1/19/2024,1/19/2024,N/A,Developed with both contracting and in-house resources.,,No,No,N/A,,,No,N/A,,,In-house curated translation memory for training and fine-tuning models,,,,,,,,,,Yes,,,,,,No,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A FOIA Web ML Document Indexer,Department of State,STATE,A/PRI/TI,Mission-Enabling (internal agency support),N/A,N/A,"Development is underway for a machine learning algorithm to generate index metadata for the documents in the Department's online Freedom of Information Act (FOIA) Library, thereby improving document classification and accuracy of search. ",Plan to have a series of prepopulated metadata,Planned,Neither,2/27/2023,N/A,N/A,N/A,N/A,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A BudgetChat AI Tool,Department of State,STATE,BP/RPBI,Mission-Enabling (internal agency support),N/A,Inputting large amounts of data from paper forms into a digital system using AI.,"BudgetChat (aka BudgetSearch) is expected to consolidate budget documents to make searching and summarizing them easier, thereby streamlining access and analysis of initiatives over multiple years and programs.","Summary of information that will be verified, reviewed, and edited by humans.",Acquisition and/or Development,Neither,7/26/2024,9/12/2024,12/1/2024,N/A,Developed with both contracting and in-house resources.,,,N/A,N/A,N/A,N/A,N/A,N/A,,,Budget documents that have been published from previous and current year. ,Yes,,,N/A,N/A,N/A,,,,Yes,,,,Yes,,,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A Travel.state.gov (TSG) Content Refinement with AI Text Editor,Department of State,STATE,CA/CST,Other,Consular Affairs,Improving the quality of written communications using AI tools.,To assist in rewriting Public Facing content on International Travel Safety & Security to improve readability.,Content rewritten in a simplified manner that will be reviewed and edited by humans. ,Acquisition and/or Development,Neither,5/1/2024,5/1/2024,N/A,N/A,,,,N/A,N/A,N/A,N/A,N/A,N/A,,,,,,,N/A,N/A,N/A,,,,,,,,,,,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A Predictive Analytics Platform,Department of State,STATE,CA/CST,Other,Consular Affairs,N/A,"Planned AI Use Cases that will be on the Predictive Analytics platform. The Predictive Analytics platform facilitates AI R&D and AI modeling, LLMs with services like Azure AI services, Azure Machine Learning, Azure Databricks, Azure Synapse and Data Science Virtual Machines etc. Individual AI Use Cases will be on this platform. ",The planned platform to provide the capacity to R&D AI Use Cases.,Acquisition and/or Development,Neither,3/15/2024,8/12/2024,N/A,N/A,Developed with both contracting and in-house resources.,,,N/A,N/A,N/A,N/A,N/A,N/A,,,,,,,N/A,N/A,N/A,,,,Yes,,,,,,,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A Consular Affairs Photo Quality Service (PQS) / FaceVACS,Department of State,STATE,CA/CST,Other,Consular Affairs,None of the above.,"To automatically check passport photo quality during the Online Passport Renewal (OPR) process, providing instant feedback to ensure submitted images meet requirements. Applicants are prompted to retake and upload new photos if needed to meet requirements or submit a physical photo through standard processes if an acceptable digital photo cannot be obtained.",Improved outcomes.,Implementation and Assessment,Neither,,,,,Developed with both contracting and in-house resources.,,,N/A,N/A,,,,N/A,,,,,,,,,,,,,,,,,,,,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A Innovation and Transformation Measurement and Prediction,Department of State,STATE,CA/CST,Other; Mission-Enabling (internal agency support),Consular Affairs,N/A,"To measure and understand causal impacts of consular service changes and improvements on desired outcomes (e.g., efficiency, effectiveness, customer and employee experiences, actionable behaviors, etc.) using methods such as machine learning and statistical modeling. This will allow Consular Affairs to better predict how these service changes will impact customers and employees before they are implemented.",Improved outcomes.,Initiated,Neither,5/1/2024,N/A,N/A,N/A,N/A,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A Evaluating Customer Feedback and Sentiments with AI,Department of State,STATE,CA/CST,Other; Mission-Enabling (internal agency support),Consular Affairs,N/A,To leverage Natural Language Processing (NLP) and secure Large Language Models (LLM) on unstructured text data to identify actionable insights to drive customer improvement initiatives. ,"Sentiment analysis, categorization, and summarization of unstructured text data.",Initiated,Neither,7/1/2024,N/A,N/A,N/A,N/A,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A Translation of Consular Content using AI,Department of State,STATE,CA/CST,Other,Consular Affairs,N/A,"To leverage AI translation models to increase CA's capacity to provide consular content in our websites to customers in their preferred language while ensuring accuracy and understanding, and reducing the resources typically needed at post.",Translated text,Planned,Neither,N/A,N/A,N/A,N/A,N/A,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A Travel.State.Gov (TSG) Enhanced Search and Chatbot ,Department of State,STATE,CA/CST,Other,Consular Affairs,N/A,A new chatbot and enhanced search on the TSG website processing existing FAQs and resources. The chatbot provides predefined responses. ,Improved outcomes.,Acquisition and/or Development,Neither,2/1/2024,2/1/2024,N/A,N/A,Developed with both contracting and in-house resources.,,,N/A,N/A,N/A,N/A,N/A,N/A,,,Data is posted on the site at www.travel.state.gov,The data is public facing and documented in www.travel.state.gov.,,,N/A,N/A,N/A,,,,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A CodeGen - AI-assisted IT Application Development,Department of State,STATE,CA/CST,Other; Mission-Enabling (internal agency support),Consular Affairs,None of the above.,"CodeGen is an application designed for seamless integration with Large Language Models (LLMs) that aims to enhance the process that developers follow to build, test, and deploy software. Through use of API, it provides a secure suite of capabilities that fast-tracks developers during code generation, translation, analysis, optimization, and debugging.",Improved outcomes.,Acquisition and/or Development,Neither,6/9/2023,8/22/2023,N/A,N/A,Developed with both contracting and in-house resources.,,,N/A,N/A,N/A,N/A,N/A,N/A,,,Application source code and documentation.,Application source code and documentation. ,,,N/A,N/A,N/A,,,,Yes,,,,,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A Violence Against Civilians Model,Department of State,STATE,CSO,Other,Conflict Early Warning,N/A,"A machine learning model that uses open-source political, social, and economic datasets to forecast mass civilian killings for the upcoming quarter and year for each country globally in order to inform conflict prevention. ",Forecasted quarterly and yearly estimates of indicators. ,Implementation and Assessment,Neither,1/23/2023,1/23/2023,1/23/2023,N/A,Developed with both contracting and in-house resources.,,No,No,N/A,,,No,No,,,,Documentation is complete,None,,No,,,,,,Yes,,Other,,Other,,,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A Automated Damage Assessments,Department of State,STATE,CSO,Diplomacy & Trade,N/A,N/A,The Conflict Observatory cooperative agreement used AI and machine learning on moderate and high-resolution commercial satellite imagery to create automated damage assessments. This project is retired.,N/A This was retired.,Retired,Neither,5/17/2022,5/17/2022,11/15/2022,8/31/2023,N/A,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A Automated Burning Detection ,Department of State,STATE,CSO,Diplomacy & Trade,N/A,N/A,Daily scans of moderate resolution commercial satellite imagery to identify anomalies using the near-infrared band. This project is retired.,N/A This was retired.,Retired,Neither,1/31/2021,1/31/2021,2/1/2022,7/31/2023,N/A,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A Senturion Alpha,Department of State,STATE,CSO,Other,Conflict Early Warning and Negotations,Searching for information using AI.,A stakeholder/influence-driven model that identifies where key decision makers fall on an issue spectrum and who influences whom. The simulation analyzes the political dynamics within contexts and estimates how the policy positions of competing interests will evolve over time.,A simulation.,Implementation and Assessment,Neither,,,,,,,,N/A,N/A,,,,N/A,,,,,,,,,,,,,,,,,,,,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A Mass Mobilization Model,Department of State,STATE,CSO,Other,Conflict Early Warning,N/A,"A machine learning forecasting model that uses open-source political, social, and economic datasets to predict mass mobilizations (protests and riots) for each country globally. ",Forecasted yearly estimates.,Implementation and Assessment,Neither,1/1/2023,1/1/2024,1/1/2024,N/A,Developed with both contracting and in-house resources.,,No,No,N/A,,,No,No,,,,Documentation is complete,None,,No,,,,,,Yes,,Other,,Other,,,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A Apptio,Department of State,STATE,DT,Mission-Enabling (internal agency support),N/A,None of the above.,Working Capital Fund (IRM/WCF) used Apptio to bill bureaus for consolidated services run from the WCF. Cost models were built in Apptio so bureaus can budget for the service costs in future Fiscal Years (FYs). Apptio had the capability to extrapolate future values using several available formulas. This project is retired.,N/A,Initiated,Neither,3/1/2023,N/A,N/A,9/29/2024,N/A,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A DT Data Analytics and Assessment (DAA) AI Use Case ITCP data harvest,Department of State,STATE,DT,Other,"Enterprise Services Directorate (ES). The Data Analytics Unit (DAU) within Diplomatic Technologies Enterprise Services provides stakeholders across the Agency with access to data, empowering them to make informed decisions. Enhancing our AI/ML capabilities within core data platforms would significan",None of the above.,To allow DT DAA team to parse unstructured text from key DT documents to build structured data indicating compliance to DT data governance and DT FAM related policies.,Plan to have structured data.,Planned,Neither,,,,,N/A,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A ServiceNow AI-Powered Virtual Agent (Chatbot),Department of State,STATE,DT/BMP/ITA/SSM,Mission-Enabling (internal agency support),N/A,Searching for information using AI.,DT (formerly IRM) BMP Systems planned to incorporate ServiceNow's Virtual Agent into existing applications to connect users with support and data requests. The Artificial Intelligence (AI) was provided by ServiceNow as part of their Platform as a Service (PaaS). This project is retired.,N/A This was retired.,Retired,Neither,7/1/2023,10/1/2022,9/30/2023,6/29/2024,N/A,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A POA&M Orchestration,Department of State,STATE,DT/DCIO/CO/A&A,Mission-Enabling (internal agency support),N/A,Identifying unusual patterns in system logs from a single incident report using AI.,This planned AI Use Case involves the orchestration of automatically opening and closing of Plan of Action and Milestones (POA&Ms) based upon log and vulnerability data specific to each Federal Information Security Modernization Act (FISMA) system.,N/A,Planned,Neither,N/A,N/A,N/A,N/A,N/A,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A NLP for Foreign Assistance Appropriations Analysis,Department of State,STATE,F,Mission-Enabling (internal agency support),N/A,Summarizing the key points of a lengthy report using AI.,Natural Language Processing (NLP) application for F/RA to streamline the extraction of earmarks and directives from the annual appropriations bill. Before NLP this was an entirely manual process.,Extracted data.,Acquisition and/or Development,Neither,,,,,Developed with both contracting and in-house resources.,,No,No,N/A,N/A,N/A,N/A,N/A,,,,,None,,N/A,N/A,N/A,,,,No,Not required,N/A,,,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A ForeignAssistance.gov Processing for Mismatched Data,Department of State,STATE,F,Other,Foreign Assistance financial data,N/A,Created a custom Natural Language Processing (NLP) model to recommend information for tagging and identify mismatched data. ,Potential mismatches and suggested tags.,Acquisition and/or Development,Neither,6/1/2023,7/24/2023,7/6/2023,N/A,Developed with both contracting and in-house resources.,,No,No,N/A,N/A,N/A,N/A,N/A,,,Foreign Assistance Obligations and Disbursements.,Yes,None,,N/A,N/A,N/A,N/A,,,No,Not required,N/A,,,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A Creating Persistent Virtual Reality Personas for dynamic training,Department of State,STATE,FSI/EX,Education & Workforce,N/A,N/A,"The EdTech Innovation Lab team plans to develop a dynamic integration between multiple AI tools to create stored/persistent ""personas"" that can be interacted with during training-related virtual reality (VR) exercises.",Plan to have persistent personas.,Planned,Neither,N/A,N/A,N/A,N/A,N/A,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A FSI Enterprise Operations - Gaming and Simulations,Department of State,STATE,FSI/EX,Education & Workforce,N/A,N/A,"In FY 2024, the bureau began a pilot program to better accommodate the need for experiential learning through gaming and simulations. Simulations can involve complicated scenarios in several different target languages. ",Plan to have simulations.,Planned,Neither,N/A,N/A,N/A,N/A,N/A,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A FSI Continuous Learning Solutions,Department of State,STATE,FSI/EX,Education & Workforce,N/A,N/A,"FSI plans to sustain our AI action plan approved by the Front Office to promote content generation, analytical tasks, and increased overall efficiency and productivity around content development.",Improved outcomes.,Planned,Neither,N/A,N/A,N/A,N/A,N/A,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A Enhancing Training Effectiveness in FSILearn Using AI,Department of State,STATE,FSI/EX/EDS,Education & Workforce,N/A,N/A,"Plan to use AI on the Cornerstone Learning Management System to enhance employee learning experiences by analyzing key data related to learning behaviors, such as course completions, quiz scores, and engagement levels. The AI uses this information to tailor learning paths.",Plan to have tailored learning paths.,Planned,Neither,N/A,N/A,N/A,N/A,N/A,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A J Reports Data Collection & Management Tool (DCT),Department of State,STATE,J,Diplomacy & Trade; Mission-Enabling (internal agency support),N/A,N/A,"The DCT is an AI-driven research assistant designed to increase the quality and maintain the integrity of select Congressional reports while reducing the labor hours it takes to generate them. The tool automates the summarization, translation, and categorization of user-submitted content into a searchable database, replacing the manual tracking and organizing of research. Users can upload materials throughout the year and easily retrieve content during drafting season. ","The DCT provides AI-generated summaries and translations of user-submitted research. It also suggests categories to organize the research according to instructions. The DCT uses optical character recognition (OCR) to digitize images and documents (e.g., PDFs).",Implementation and Assessment,Neither,4/1/2023,6/20/2023,11/29/2023,N/A,Developed with both contracting and in-house resources.,,No,No,N/A,,,No,No,,,"DCT Evaluation Data: An internal research dataset of user-submitted articles, reviewed and labeled by subject matter experts to facilitate model evaluation.",Documentation is widely available,None,,Yes,,,Data.State-SBU,Data.State-SBU,,Yes,,Yes,,Yes,,,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A TIP Report Research Translation,Department of State,STATE,J/TIP,Mission-Enabling (internal agency support),N/A,None of the above.,"The TIP Report Translation AI system provides informal, unofficial translations for materials related to the annual Trafficking in Persons report. This capability saves staff time and allows researchers and drafters to focus on critical report needs, such as identifying and resolving data gaps. ","Informal, unofficial translations of PDFs and Microsoft word documents. Each document includes a watermark denoting that the AI translation is unofficial. Each document must be verified, reviewed, and edited by a human.",Operation and Maintenance,Neither,4/1/2023,6/20/2023,9/1/2023,N/A,,,,N/A,N/A,,,,N/A,,,,,,,,,,,,,,,,,,,,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A FOIA 360 AI Matching Tool,Department of State,STATE,M/SS/CFA,Other,Government Transparency,N/A,This system aims to make the Department of State's FOIA process better by spotting similar requests and documents to cut down on duplication. It also helps find patterns in both current and new requests to speed up response times for the public.,Improved outcomes.,Implementation and Assessment,Neither,3/2/2023,6/5/2023,4/1/2024,N/A,Developed with both contracting and in-house resources.,,No,No,N/A,No,N/A,No,No,,N/A,FOIA request text from FOIA Express and documents captured from the Department's Virtual FOIA Reading Room are used for training and testing,Documentation is complete,None,N/A,Yes,"Yes – agency has access to source code, but it is not public.",N/A,Yes,ElasticSearch,Less than 6 months,Yes,N/A,Yes,N/A,Yes,N/A,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A StateChat,Department of State,STATE,M/SS/CFA,Mission-Enabling (internal agency support),N/A,N/A,StateChat is the Department's enterprise Generative AI-powered chatbot.,"StateChat is a chatbot, its outputs are textual in a chat interface.",Implementation and Assessment,Neither,1/1/2024,2/1/2024,3/1/2024,N/A,Developed with both contracting and in-house resources.,,No,No,N/A,No,N/A,Yes,Yes,Yes,N/A,The model is not trained or fine-tuned with agency data; it is provided various types of agency data as context. ,Documentation is widely available,None,N/A,Yes,,N/A,Yes,StateChat,Less than 6 months,Yes,N/A,Yes,N/A,Yes,N/A,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A Crisis Campaign Cable Analytics,Department of State,STATE,M/SS/CFA,Mission-Enabling (internal agency support),N/A,N/A,Used optical character recognition and Natural Language Processing (NLP) on Department cables in order to evaluate gaps and trends in crisis training and bolster post preparedness for crisis events. This project is retired.,N/A This was retired.,Retired,Neither,1/16/2023,4/3/2023,5/15/2023,5/31/2023,N/A,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A NLP to pull key information from unstructured text,Department of State,STATE,PM,Mission-Enabling (internal agency support),N/A,N/A,Natural language processing (NLP) to pull key information from unstructured text. Use NLP to extract information such as country names and agreement dates from dozens of pages of unstructured PDF document.,Dataset with key information from unstructured data.,Implementation and Assessment,Neither,4/1/2022,4/15/2022,9/30/2022,N/A,Developed with both contracting and in-house resources.,,No,No,N/A,,,No,No,,,,Documentation is complete,None,,,,,N/A,,Less than 6 months,,,,,Other,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A ECA Program Management and Outreach - Summarization,Department of State,STATE,R/ECA/EX/IT,Mission-Enabling (internal agency support),N/A,None of the above.,Use approved Large Language Models to create artifacts for leadership as well as visiting legateship utilizing relevant and accurate datasets.,Plan to have draft summaries that will be reviewed by humans.,Planned,Neither,N/A,N/A,N/A,N/A,N/A,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A Storyzy,Department of State,STATE,R/GEC,Other,Synthetic media detection,None of the above.,Improve detected use of synthetic content.- Synthetic content is computer-generated data that mimics real-world data.,Improved outcomes.,Operation and Maintenance,Neither,6/1/2023,7/17/2023,12/1/2023,N/A,Developed with both contracting and in-house resources.,,No,No,N/A,,,No,No,,,Open source data from social media.,No,None,,No,,,N/A,,,,,,,Other,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A Optical Character Recognition - Text Extraction,Department of State,STATE,R/GEC,Diplomacy & Trade,N/A,N/A,Extracted text from images using standard python libraries; inputs were websites to collect data. This project is retired.,N/A This was retired.,Retired,Neither,9/21/2021,9/21/2021,12/21/2021,6/30/2023,N/A,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A Topic Modeling,Department of State,STATE,R/GEC,Diplomacy & Trade,N/A,N/A,Cluster text into themes based on frequency of used words in documents; was applied to digital media articles as well as social media posts; performed using available Python libraries. This project is retired.,N/A This was retired.,Retired,Neither,,,,6/30/2023,N/A,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A Forecasting,Department of State,STATE,R/GEC,Diplomacy & Trade,N/A,N/A,"Using statistical models, projecting expected outcomes into the future; this was applied to COVID cases as well as violent events in relation to tweets. This project is retired.",N/A This was retired.,Retired,Neither,,,,6/30/2023,N/A,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A Deepfake Detector,Department of State,STATE,R/GEC,Diplomacy & Trade,N/A,N/A,"Deep learning model that took in an image containing a person's face and classifies the image as either being real (contains a real person's face) or fake (synthetically generated face, a deepfake often created using Generative Adversarial Networks). This project is retired.",N/A This was retired.,Retired,Neither,6/21/2021,6/21/2021,9/21/2021,6/30/2023,N/A,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A SentiBERTIQ,Department of State,STATE,R/GEC,Diplomacy & Trade,N/A,N/A,"Used deep contextual AI of text to identify and extract subjective information within the source material. This sentiment model was trained by fine-tuning a multilingual, BERT model leveraging word embeddings across 2.2 million labeled tweets spanning English, Spanish, Arabic, and traditional Chinese. This project is retired.",N/A This was retired.,Retired,Neither,,,,6/30/2023,N/A,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A TOPIQ,Department of State,STATE,R/GEC,Diplomacy & Trade,N/A,N/A,"Automatically categorized text into topics for analyst review and interpretation, using Latent Dirichlet Allocation (LDA), to extract topics from documents and then assigned the probability that each document belonged to a topic. This project is retired.",N/A This was retired.,Retired,Neither,,,,6/30/2023,N/A,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A Text Similarity,Department of State,STATE,R/GEC,Diplomacy & Trade,N/A,N/A,Identified different texts that were identical or nearly identical by calculating cosine similarity between each text. Texts were then grouped if they share high cosine similarity and then available for analysts to review further. This project is retired.,N/A This was retired.,Retired,Neither,,,,6/30/2023,N/A,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A Image Clustering,Department of State,STATE,R/GEC,Diplomacy & Trade,N/A,N/A,"Used a pretrained deep learning model to generate image embeddings, then used hierarchical clustering to identify similar images. This project is retired.",N/A This was retired.,Retired,Neither,,,,6/30/2023,N/A,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A Louvain Community Detection,Department of State,STATE,R/GEC,Diplomacy & Trade,N/A,N/A,"Took in a social network and clusters nodes together into ""communities"" (i.e., similar nodes are grouped together). This project is retired.",N/A This was retired.,Retired,Neither,,,,6/30/2023,N/A,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A Fast Text Word Builder,Department of State,STATE,R/GEC,Diplomacy & Trade,N/A,N/A,Fast Text was an AI approach to identifying similar terms and phrases based on a root word. This supported A&R's capability to build robust search queries for data collection. This project is retired.,N/A This was retired.,Retired,Neither,3/15/2023,4/1/2023,4/1/2023,6/30/2023,N/A,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A Digital Media Analytics Platform,Department of State,STATE,R/GPA/RA,Mission-Enabling (internal agency support),N/A,N/A,Using open-source neural machine translation models to translate global media articles and Department and public foreign social media posts into English. ,Trend analysis,Implementation and Assessment,Neither,11/1/2022,12/1/2022,1/1/2024,N/A,Developed with both contracting and in-house resources.,,,,N/A,,,,,,,,,None,,Yes,"Yes – agency has access to source code, but it is not public.",,,,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A Geospatial Mapping of Historical Survey Data,Department of State,STATE,R/GPA/RA,Mission-Enabling (internal agency support),N/A,Creating visual representations of data sets for reports and presentations using AI.,Used PowerBI to create an interactive map. Cleaned and collected historical data consisting of all statistically significant results from all Pollfish surveys/tests. This project is retired.,N/A This was retired.,Retired,Neither,1/1/2022,4/1/2022,5/1/2022,12/31/2022,N/A,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A AI Tools to Enhance PD Workflows,Department of State,STATE,R/GPA/RA,Mission-Enabling (internal agency support),N/A,Improving the quality of written communications using AI tools.,Plan to leverage ChatGPT Licenses to streamline existing Public Diplomacy workflows with AI tools following Department policy.,N/A,Planned,Neither,N/A,N/A,N/A,N/A,N/A,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A LEO - Budget Office Inquiries,Department of State,STATE,WHA/EX,Mission-Enabling (internal agency support),N/A,N/A,Answer questions frequently asked about budget. ,Answers to common questions.,Acquisition and/or Development,Neither,1/30/2024,9/5/2024,5/1/2024,N/A,Developed with both contracting and in-house resources.,,,N/A,N/A,N/A,N/A,N/A,N/A,,,,,,,N/A,N/A,N/A,,,,,,,,Yes,,,,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A Collections Chatbot,Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,None of the above.,"This project implemented the SBSE chatbot, which provides self-service Frequently Asked Questions (FAQ) to taxpayers regarding payments, payment plans, financial relief and offer in compromise topics. Three new topics are currently in development and scheduled to be incorporated into the chatbot on 3/13/2025: Automated Substitute for Return (ASFR), Centralized Lien Operation (CLO), and Withholding Compliance (WHC). The Intent Engine (using Natural Language Processing (NLP) and AI algorithms) classifies taxpayers' utterances (questions) into intents (requests). The answers provided to the taxpayers are not created by Generative AI. The responses of the chatbot and business logic is predetermined by content owners. This AI tool aims to classify/navigate to a correct predetermined response.","The NLP algorithm functions by classifying user inputs into predefined topics using advanced natural language processing techniques. It evaluates the input against a set of trained models and assigns a response based on a confidence score calculated by the algorithm. Only responses associated with these pre-established topics are provided, ensuring accuracy and consistency. The algorithm does not generate new or ad-hoc answers outside the defined topics, making it reliable for controlled environments where precision and adherence to predetermined content are critical.",Operation and Maintenance,Neither,4/1/2021,6/1/2021,12/10/2021,,Developed with contracting resources.,Contract # NNG15SD27B,No,,,No,,No,Yes,Yes,,None,"Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",None,,No,No – agency does not have access to source code.,,Yes,eGain Suite,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, Systran Translation Software,Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,None of the above.,"Systran (SYSTRAN Pure neural Server) language translation software provides language translation capability. It is a Commercial Off-the-Shelf (COTS) product, and version 9 is deployed. Users only use their documentation that is in either English or Non-English, and Systran will translate the content to target language. No documents are stored in Systran. Users are typically IRS staff requiring translation of Non-English to English for cursory evaluation (Triage) of relevancy to case work. Some users are translating English to a Non-English language as a beginning for manual translation. The Neural Network based translation engine can be augmented with dictionary and translation memories to bolster domain specific language accuracy, and dictionary created during a MITRE task order was added in c. 2020.",Translated content (in target language picked by user) of the source (input) text submitted by the user. User then can read and review the translation on either a graphical user interface (GUI) field or export. ,Operation and Maintenance,Neither,9/1/2020,9/1/2020,9/1/2020,,Developed with contracting resources.,2043FY-22-F-00078 P00002,No,,,No,,No,Yes,No,,Publication 850 and other official translation pairs received from IRS' Human Translation team were used to add this information as translation memories data source to base translation engine. All such information is non-sensitive. ,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups.",None,,No,No – agency does not have access to source code.,,Yes,RAS-1,Less than 6 months,No,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, Automated Underreporter (AUR) Voicebot,Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,None of the above.,"The AUR Voice Bot will authenticate taxpayers using “Shared Secrets” and answer questions requiring account look ups and/or updates. It will give answers to provide confirmation we received their response, the current status of their case, and grant requests for additional time to respond if they are eligible.",The system produces a predefined response that provides taxpayers with information related to their account status or current case details after authentication. It also helps route calls to appropriate service areas or escalate to live agents when necessary.,Operation and Maintenance,Neither,2/9/2023,9/25/2023,5/22/2024,,Developed with contracting resources.,2032H5-19-F-00641,No,,,No,,No,Yes,Yes,,"Uses historical taxpayer interaction data, such as account inquiries and case resolution logs, to train and evaluate performance. Metadata from AUR case files and voice interaction transcripts.","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups.",None,,No,No – agency does not have access to source code.,,Yes,Nuance Digital Contact Center Platform (DCCP),Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, Employee Resource Center (ERC) Chatbot,Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,None of the above.,"This project implemented the ERC chatbot, which provides self-service FAQs to IRS employees regarding administrative support, benefit topics, credit card services, HRConnect questions, retirement, insurance, and various other topics related to the ERC. The Intent Engine (using NLP and AI algorithms) classifies users' utterances (questions) into intents (requests). The answers provided to the taxpayers are not created by Generative AI. The responses of the chatbot and business logic is predetermined by content owners. This AI tool aims to classify/navigate to a correct predetermined response.","The NLP algorithm functions by classifying user inputs into predefined topics using advanced natural language processing techniques. It evaluates the input against a set of trained models and assigns a response based on a confidence score calculated by the algorithm. Only responses associated with these pre-established topics are provided, ensuring accuracy and consistency. The algorithm does not generate new or ad-hoc answers outside the defined topics, making it reliable for controlled environments where precision and adherence to predetermined content are critical.",Operation and Maintenance,Neither,9/30/2022,3/1/2023,7/20/2023,,Developed with contracting resources.,Contract # NNG15SD27B,No,,,No,,No,Yes,Yes,,Project utilized data provided by content Subject Matter Experts to evaluate the accuracy of responses.,"Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",None,,No,No – agency does not have access to source code.,,Yes,eGain Suite,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, Robotic Process Automation for Form 941-X Extraction,Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,None of the above.,"What is the purpose/objective of this AI project? Intelligent Data extraction from scanned 941X forms At a high level, how does it accomplish its purpose? Digital processing of Form 941X using Artificial Intelligence and Intelligent Document Processing to read scanned forms, extract data in usable format and distribute the data to downstream systems. What is the decision or judgment that this model automates? Learning and improving character recognition while extracting the data. Decision is about recognizing the extracted character correctly. Are humans in the loop in reviewing the AI output prior to taking action on the results? If so, how? Yes, When the character extraction confidence scores are lower, the form is diverted to UiPath Action Center, where a Human in the loop reviews the extracted data and confirms/corrects before submitting. What benefits are anticipated to come from this project? Speed, volume, and Accuracy in data extraction from scanned taxpayer forms What is the direct or ultimate impact of this decision? Speed of data extraction from paper forms and clearing the backlog of 941x forms to be processed. What is the population impacted by this decision? Data Transcribes can transcribe a lot more forms at a much faster rate than manual process.",The system will extract text from the forms (pdf's) translating handwritten text into a digital format. After the extraction process the system presents what was extracted to the end user to then validate before anything is sent to any downstream systems.,Operation and Maintenance,Neither,9/11/2023,9/11/2023,11/20/2023,,Developed with both contracting and in-house resources.,2032H5-21-F-00495 Blue Tech Number / 2032H5-22-F-00922 P00003 Accenture Contract ,No,,,No,,No,Yes,No,,"The forms (Form 940, Form 941, and Form 1040) are owned by the IRS.","Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,Yes,No – agency does not have access to source code.,,Yes,Robotic Process Automation Platform,Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Taxpayer Services Chatbot,Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,None of the above.,"This project implemented the W&I chatbot, which provides self-service Frequently Asked Questions (FAQ) to taxpayers regarding refunds and Advanced Child Tax Credit topics. The Intent Engine (using Natural Language Processing (NLP) and AI algorithms) classifies taxpayers' utterances (questions) into intents (requests). The answers provided to the taxpayers are not created by Generative AI. The responses of the chatbot and business logic is predetermined by content owners. This AI tool aims to classify/navigate to a correct predetermined response.","The NLP algorithm functions by classifying user inputs into predefined topics using advanced natural language processing techniques. It evaluates the input against a set of trained models and assigns a response based on a confidence score calculated by the algorithm. Only responses associated with these pre-established topics are provided, ensuring accuracy and consistency. The algorithm does not generate new or ad-hoc answers outside the defined topics, making it reliable for controlled environments where precision and adherence to predetermined content are critical. ",Operation and Maintenance,Neither,2/1/2022,6/9/2022,3/14/2023,,Developed with contracting resources.,Contract # NNG15SD27B,No,,,No,,No,Yes,Yes,,Project utilized data provided by content Subject Matter Experts to evaluate the accuracy of responses.,"Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",None,,No,No – agency does not have access to source code.,,Yes,eGain Suite,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, Winnie Chatbot for Employee IT FAQs,Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,None of the above.,"This project implemented the Winnie chatbot, which provides self-service Frequently Asked Questions (FAQ) to IRS staff regarding various Information Technology (IT) topics. The Intent Engine (using Natural Language Processing (NLP) and AI algorithms) classifies users' utterances (questions) into intents (requests). The answers provided to the users are not created by Generative AI. The responses of the chatbot and business logic is predetermined by content owners. This AI tool aims to classify/navigate to a correct predetermined response.","The NLP algorithm functions by classifying user inputs into predefined topics using advanced natural language processing techniques. It evaluates the input against a set of trained models and assigns a response based on a confidence score calculated by the algorithm. Only responses associated with these pre-established topics are provided, ensuring accuracy and consistency. The algorithm does not generate new or ad-hoc answers outside the defined topics, making it reliable for controlled environments where precision and adherence to predetermined content are critical.",Operation and Maintenance,Neither,12/5/2022,9/26/2023,9/26/2023,,Developed with contracting resources.,Contract # NNG15SD27B,No,,,No,,No,Yes,No,,Project utilized data provided by content Subject Matter Experts to evaluate the accuracy of responses.,"Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",None,,No,No – agency does not have access to source code.,,Yes,eGain Suite,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, DATA Act Bot for Procurement Data Matching,Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,None of the above.,"Purpose: The Digital Accountability and Transparency Act (DATA) Act Bot automates verifying that IRS Federal Procurement Data System (FPDS) reporting matches the information in contract documents (e.g. dollar amounts, dates, location of work). Supervised learning and natural language processing are used to extract unstructured information from contract documents. F1 accuracy scores are used to measure performance of validation models for each specific data element. Benefits: The AI improves operational efficiency by validating consistency and accuracy of contract metadata.","AI model validates that contract spending information reported to USASpending.gov matches contract documents. The system validates the consistency of contract metadata such as contract number, modification number, dollar amounts, contract work / place of performance location address, and contract dates.",Operation and Maintenance,Neither,9/26/2019,9/26/2019,3/31/2022,,Developed with contracting resources.,2032H822F00222,No,,,No,,No,Yes,Yes,,"IRS contract documents were used to train the model. Project team members manually annotated or created labeled data. Created dataset of information extracted from contract documents (e.g. dates, dollar amounts, addresses). This dataset is used to improve the accuracy and reliability of contract spending reporting.","Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Integrated Financial System / Procurement for the Public Sector,More than 12 months,Other,AI component is a contractor managed service application accessed using bot running on agency infrastructure.,No,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, Taxpayer Services Form 1040 Voicebot,Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,None of the above.,"This project is implementing 1040 IVR voice bot for Taxpayer Sevices (TS), formally known as Wage & Investments (W&I), which allows them to check their individual tax information, such as Late Filing Season Individual Income Tax Return Menu, Non-Filing Season Individual Income Tax Return Menu. Taxpayer could be directed to the appropriate Customer Service Representative based on their request. . The Intent Engine (using NLP and AI algorithms) classifies taxpayers' utterances (questions) into intents (requests). The responses of the voicebot and business logic is predetermined by content owners. This AI tool aims to classify/navigate to a correct predetermined response.",Provide caller with information based on business logic/rules.,Operation and Maintenance,Neither,2/21/2023,11/13/2023,3/5/2024,,Developed with contracting resources.,2032H5-19-F-00641,No,,,No,,No,Yes,Yes,,Call summary/call detail records,"Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",None,,No,No – agency does not have access to source code.,,Yes,Nuance Digital Contact Center Platform (DCCP),Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, Where's My Refund and Where's My Amended Return Voicebot,Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,None of the above.,"This project is implementing Where's My Refund (WMR) and Where's My Ammended Return (WMAR) voice bots for Accounts Management (AM). WMR voicebot allows taxpayers to obtain the status of their refund and fact of filing information. Taxpayers can initiate a Refund Trace, or re-issuance of their refund check, for their current year refunds. WMAR voicebot allows taxpayers to request the status of their amended tax return. The Intent Engine (using NLP and AI algorithms) classifies taxpayers' utterances (questions) into intents (requests). The answers provided to the taxpayers are not created by Generative AI. The responses of the voicebot and business logic is predetermined by content owners. This AI tool aims to classify/navigate to a correct predetermined response.",Provider caller with status on their WMR/WMAR.,Operation and Maintenance,Neither,4/1/2022,3/7/2023,1/4/2024,,Developed with contracting resources.,2032H5-19-F-00641,No,,,No,,No,Yes,Yes,,Call summary and call detail data.,"Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",None,,No,No – agency does not have access to source code.,,Yes,Nuance Digital Contact Center Platform (DCCP),Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, Simulation of the Nation (SimoN) Synthetic Data Engine,Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,None of the above.,"The Internal Revenue Service (IRS) Enterprise Services (ES), Enterprise Systems Testing (EST) is the testing authority for Information Technology (IT) and provides enterprise-wide testing solutions and critical support across the enterprise for numerous systems and applications. The Synthetic Data Engine, or Simulation of the Nation (SimoN) is designed to generate volumes of synthetic tax data to represent a wide range of use cases, without copying Federal Tax Information (FTI) from production systems is an IRS directive. SimoN is also able to age data to support multi-year tax code concerns for more comprehensive testing sequences spanning multiple tax years. This will improve the volume, diversity and quality of test data being created and processed in a systemic manner. To date, SimoN has produced over 700,000 unique and valid Individual Master File (IMF) tax returns, and over 50,000 unique and valid Business Master File (BMF) returns for testing. SimoN also supports Social Security Administration (SSA) interconnects and SS4 interconnects which allow the data produced to be fully linked between business and individuals. Over 320 dimensions of socioeconomic data are synthesized and validated against United States (U.S.) Census metrics to ensure a validated and realistic synthetic dataset. The SimoN solution consumes publicly available statistical data about businesses and the population of the U.S. This includes tax household composition, income and education levels and other data points. The solution combines statistical data with Artificial Intelligence (AI) and Machine Learning (ML) to generate businesses and simulated households with a variety of tax scenarios. The solution then stores the business and population data in a database. The data is extracted to files for submission to the test environment systems simulating eFile and SSA feeds with no review of data created. At no point, is Production data (i.e., Personally Identifiable Information (PII), Federal Taxpayer Information (FTI) or Controlled Unclassified Information (CUI) used as input to the Artificial Intelligence/Machine Language (AI/ML) process of SimoN, nor does the simulated output data feed Production systems. In addition to generating synthetic data, this system is designed to simulate the aging of the population. The aging model includes a variety of socioeconomic events, such as births, deaths, marriages, job changes and address changes. SimoN also has multiple levels of automated testing and data validations to ensure the data produced is usable by IRS test systems and simulates businesses and population.","SimoN is capable of outputting 3 tax years of IMF and BMF returns starting in Tax Year 2022. With each XML Schema Definition (XSD) that is released under the Modernized e-File (MeF), SimoN implements each XSD version and provides full regression support for testing in successive years. Each XSD version has automated tests that run to ensure verifications can be run to reduce the likelihood of AI generated synthetic returns that might have data anomalies. Also, to help seed test systems with synthetic individuals and synthetic businesses, SimoN can output the Data Master 1 (DM1) file simulating a feed from Social Security Administration (SSA) and Application for Employer Identification Number file (Form SS-4) containing transcribed form information so that the test system can be initialized with the appropriate backend reference data for validation of IMF and BMF entities, as well as the ability to validate incoming synthetic tax returns.",Operation and Maintenance,Neither,4/4/2021,10/1/2021,1/31/2024,,Developed with contracting resources.,2032H5-22-C-00097,No,,,No,,No,Yes,No,,"None, all data used for unsupervised model training is based on public data sets from the Census Bureau.","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups.","Marital Status, Residency Status, Employment Status, Income, Other, Sex/Gender, Age, Socioeconomic Status, Ability Status",SimoN leverages over 100 different census attributes for model training in order to generate a synthetic population.,Yes,"Yes – agency has access to source code, but it is not public.",,No,,6-12 months,Yes,,No,,No,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Machine Translation,Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,None of the above.,"To better facilitate and automate content translation efforts at the IRS, a cloud-based commercial off-the-shelf (COTS) solution is being leveraged for use by the Linguistic, Policy, Tools and Services (LPTS) Team. The Machine Translation (MT) application leverages AWS Neural Translate and is being assessed and evaluated by the primary users, the Linguistic Policy, Tools and Services (LPTS) organization, through integration with their existing processes. The MT application focuses on translation for existing text and labels into Spanish with the ultimate goal of becoming an enterprise solution for a variety of non-English translations.",The MT application focuses on translation for existing text and labels into Spanish and other languages with the ultimate goal of becoming an enterprise solution for a variety of non-English translations.,Operation and Maintenance,Neither,3/1/2022,10/3/2022,9/25/2023,,Developed with contracting resources.,"2032H5-22-P-00059,2032H5-24-F-00147",No,,,No,,No,Yes,No,,Custom IRS Pub 850 glossary of English and Spanish tax terms and phrases developed by the IRS is periodically uploaded to allow the MT Application to provide the most consistent translations possible.,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups.",None,,No,"Yes – agency has access to source code, but it is not public.",,Yes,Integrated Enterprise Portal,6-12 months,Yes,,Other,MT application uses Amazon Web Services (AWS) Application Programming Interfaces (APIs).,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, Economic Impact Payment (EIP) FAQ Voicebot,Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,None of the above.,"This project developed self-service application that responds to general taxpayer inquiries regarding the Economic Impact Payment (EIP) and includes options to route the call to live assistor during normal business hours. The Intent Engine/AI System (using NLP and AI algorithms) classifies taxpayers' utterances (questions) into intents (requests). The answers provided to the taxpayers are not created by Generative AI. The responses of the voicebot and business logic is predetermined by content owners. This AI tool aims to classify/navigate to a correct predetermined response.",Provide caller with information on Economic Impact Payment (EIP).,Operation and Maintenance,Neither,2/1/2021,4/9/2021,5/5/2021,,Developed with contracting resources.,2032H5-19-F-00641,No,,,No,,No,Yes,Yes,,Call summary and call detail data.,"Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",None,,No,No – agency does not have access to source code.,,Yes,Nuance Digital Contact Center Platform (DCCP),Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, One-Time Payment Voicebot,Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,None of the above.,"This project developed self-service application that responds to general taxpayer inquiries regarding the One Time Payment (OTP) and includes options to route the call to live assistor during normal business hours. The Intent Engine (using NLP and AI algorithms) classifies taxpayers' utterances (questions) into intents (requests). The answers provided to the taxpayers are not created by Generative AI. The responses of the voicebot and business logic is predetermined by content owners. This AI tool aims to classify/navigate to a correct predetermined response.",The system generates predefined responses that address common taxpayer inquiries about one-time payments. It also provides routing to live agents during business hours if the taxpayer's issue cannot be resolved by the voicebot.,Operation and Maintenance,Neither,4/1/2021,6/1/2021,1/4/2022,,Developed with contracting resources.,2032H5-19-F-00641,No,,,No,,No,Yes,Yes,,Training data includes logs from prior taxpayer inquiries regarding one-time payments and payment plans. ,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups.",None,,No,No – agency does not have access to source code.,,Yes,Nuance Digital Contact Center Platform (DCCP),Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, FAQs / Notice Clarifications Voicebot,Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,None of the above.,"This project developed self-service application that responds to general taxpayer inquiries regarding the Notice Clarifications and includes options to route the call to live assistor during normal business hours. The Intent Engine (using NLP and AI algorithms) classifies taxpayers' utterances (questions) into intents (requests). The answers provided to the taxpayers are not created by Generative AI. The responses of the voicebot and business logic is predetermined by content owners. This AI tool aims to classify/navigate to a correct predetermined response.","The system produces predefined answers to taxpayer questions about notices and other clarifications. If the question cannot be resolved automatically, the system escalates to a live agent or prompts the taxpayer for further clarification. ",Operation and Maintenance,Neither,4/1/2021,6/1/2021,1/4/2022,,Developed with contracting resources.,2032H5-19-F-00641,No,,,No,,No,Yes,Yes,,The model uses interaction data from FAQs and Notice clarifications previously provided by live agents. Metadata from taxpayer inquiries about IRS notices and FAQs.,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups.",None,,No,No – agency does not have access to source code.,,Yes,Nuance Digital Contact Center Platform (DCCP),Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, Advance Child Tax Credit (AdvCTC) Voicebot,Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,None of the above.,"This project developed self-service application that answers common taxpayer questions about the Advance Child Care Tax Credit. The Intent Engine (using NLP and AI algorithms) classifies taxpayers' utterances (questions) into intents (requests). The answers provided to the taxpayers are not created by Generative AI. The responses of the voicebot and business logic is predetermined by content owners. This AI tool aims to classify/navigate to a correct predetermined response.",The system generates predefined responses that address common taxpayer inquiries about Advance Child Care Tax Credit. It also provides routing to live agents during business hours if the taxpayer's issue cannot be resolved by the voicebot.,Operation and Maintenance,Neither,11/17/2021,11/17/2021,2/18/2022,,Developed with contracting resources.,2032H5-19-F-00641,No,,,No,,No,Yes,Yes,,Call summary and call detail data.,"Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",None,,No,No – agency does not have access to source code.,,Yes,Nuance Digital Contact Center Platform (DCCP),Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, Small Business/Self Employed (SBSE) FAQ Voicebot,Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,None of the above.,"This project developed self-service application that resides on ACI platform which answers general taxpayer inquiries regarding the One Time Payment (OTP), the Notice Clarifications and includes options to route the call to live assistor during normal business hours. The Intent Engine (using NLP and AI algorithms) classifies taxpayers' utterances (questions) into intents (requests). The answers provided to the taxpayers are not created by Generative AI. The responses of the voicebot and business logic is predetermined by content owners. This AI tool aims to classify/navigate to a correct predetermined response.",The system outputs include predefined responses to FAQs related to notices and one-time payments. It routes call to live agents during business hours if necessary and provides guidance based on the identified intent of the taxpayer. ,Operation and Maintenance,Neither,1/27/2023,6/15/2023,8/4/2023,,Developed with contracting resources.,2032H5-19-F-00641,No,,,No,,No,Yes,Yes,,The model uses data from FAQs and historical inquiries handled by live agents.,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups.",None,,No,No – agency does not have access to source code.,,Yes,Nuance Digital Contact Center Platform (DCCP),Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, Insolvency and Offer in Compromise Voicebot,Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,None of the above.,"This project developed self-service application that responds to general taxpayer inquiries regarding the OIC Program as well as insolvent taxpayers with their most requested services with the options to route the call to live assistor during normal business hours. The Intent Engine (using NLP and AI algorithms) classifies taxpayers' utterances (questions) into intents (requests). The answers provided to the taxpayers are not created by Generative AI. The responses of the voicebot and business logic is predetermined by content owners. This AI tool aims to classify/navigate to a correct predetermined response.","The outputs consist of responses to questions about Offers in Compromise (OIC) (deployed 5/31/24) and Insolvency (deployed 6/14/24), based on predefined FAQs. When the system cannot resolve an inquiry, it either prompts the user for clarification or routes the call to a live assistor.",Operation and Maintenance,Neither,12/6/2023,2/1/2024,5/31/2024,,Developed with contracting resources.,2032H5-19-F-00641,No,,,No,,No,Yes,Yes,,Data from taxpayer interactions about insolvency options and Offers in Compromise (OIC) queries. Metadata from insolvency-related queries and OIC case files. ,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups.",None,,No,No – agency does not have access to source code.,,Yes,Nuance Digital Contact Center Platform (DCCP),Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, Financial Relief Voicebot,Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,None of the above.,"This project developed self-service application that responds to general taxpayer inquiries regarding the Financial Relief and includes options to route the call to live assistor during normal business hours. The Intent Engine (using NLP and AI algorithms) classifies taxpayers' utterances (questions) into intents (requests). The answers provided to the taxpayers are not created by Generative AI. The responses of the voicebot and business logic is predetermined by content owners. This AI tool aims to classify/navigate to a correct predetermined response.","The system generates predefined answers to inquiries about financial relief options for taxpayers. The system navigates through predefined responses, asks clarifying questions if needed, and routes calls to live agents for complex issues.",Acquisition and/or Development,Neither,1/8/2024,2/1/2024,,,Developed with both contracting and in-house resources.,2032H5-19-F-00641,No,,,,,,,Yes,,Historical data from financial relief inquiries and associated case resolutions. Metadata from taxpayer requests for financial relief and payment plans. ,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups.",None,,,,,Yes,Nuance Digital Contact Center Platform (DCCP),Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, Generative AI Toolbox for Contract Document Assistance,Department of the Treasury,TREAS,Internal Revenue Service,Government Services (includes Benefits and Service Delivery),,Summarizing the key points of a lengthy report using AI.,"Exploring / planning potential use of generative AI to assist with contract document tasks such as: - Assisting with drafting contract statements of work. - Reviewing contracts and supporting documents for completeness, consistency, and policy compliance. - Reducing barriers for small businesses / new entrants to bid on Treasury contracts by offering AI powered proposal writing assistance.","The AI system generates accurate, contextually relevant, and human readable text outputs based on user input queries and uploaded documents. Its outputs include: Document Summaries., Draft Responses, Content Rewrite, Question-Answer Pairs, Data Extraction, Analysis and Recommendations.",Acquisition and/or Development,"Rights-Impacting ",,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Integrate Google Doc AI for Digitalization,Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,None of the above.,"This Digital Enablement Platform (DEP) implementation has the goal to increase access to high quality data extraction, reduce reliance on paper and manually intensive processes utilizing artificial intelligence. Therefore, eliminating the need for transcription and manual data corrections by the Business. DEP Artificial Intelligence (AI) will include updates to the existing business unit processes to leverage the UiPath Robotic Process Automation (RPA) platform. The UiPath RPA platform will also integrate with Google Document AI to implement a new extraction engine that will work in conjunction with and in support of the DEP. The UiPath Automation Suite software for RPA platform in the Integrated Enterprise Portal (IEP) Amazon Web Services (AWS) provides support for multiple optical character recognition (OCRs), included the native UiPath OCR engine for intelligent document processing. Eliminating the need for transcription and manual data corrections by the Business. Humans in the loop will validate AI OCR classification and extraction within the UiPath Action Center. Any extraction confidence levels that fail both UiPath and Google DocAI confidence level requirements necessitate human verification as well as 10 percent of all documents that are processed through the DEP AI system. The benefit of the DEP AI system revolves around the of transition to being a modern, digitally capable, customer-centric agency. It will ensure taxpayers file accurate returns as well as allowing for the IRS to identify inconsistencies and streaming correspondence. The DEP AI aims to enhance efficiency by transitioning to end-to-end digital processes, eliminating paper backlogs, and improving accuracy in translating return data. These digital tools will be accessible to individuals with disabilities and offered in taxpayers’ preferred languages. Additionally, this commitment to digitalization reduces the environmental impact by minimizing paper usage and storage. Overall, the decision benefits taxpayers, improves operations, and contributes to environmental conservation efforts. IRS business units tied to paper-driven processes as well as U.S. taxpayers.",The output is a metadata file that contains the data that has been extracted from the images we have scanned.,Implementation and Assessment,Neither,11/20/2023,11/20/2023,6/30/2024,,Developed with contracting resources.,TIRNO-17-D-00004 WR2730,No,,,No,,No,Yes,Yes,,"We have used CP2000 letter responses, Form 709 tax data, Form 94X tax data, SCOIC (offer in compromise) data.","Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Digital Enablement Platform,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, Modernization Accelerator - Legacy Applications Chatbot & Code Conversion,Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,None of the above.,"The IRS Enterprise has many systems that use a legacy coding language, which is much older than typical modern coding languages. This makes it difficult to find skilled and knowledgeable people to maintain or modernize applications. A large language model (LLM) can help speed up this translation so we can more rapidly modernize while retaining the functional objectives of the legacy code. This use case will consist of an iterative multi-phase approach to help accelerate the modernization of sample legacy code by building a Generative AI (GenAI) Sandbox environment that will allow us to ingest a subset of IRS legacy documentation to build a LLM GenAI chatbot. This will accelerate knowledge discovery for future legacy modernization initiatives. In addition, we will leverage GenAI to ingest a subset of IRS Assembler Language Code (ALC) legacy code to evaluate, validate, and optimize GenAI capabilities to not only transform legacy code into English readable pseudocode, and convert it into a modern language. This initial use case will be limited to IRS use only within a sandbox environment and not intended for production. Existing Code to Java LLM within IRS boundary","The AI accelerator outputs a document that describes the functionality of the ALC code ingested; identifying loops and conditions as well as key architectural and functional diagrams that can assist a developer understanding complex legacy code. The Retrieval Augmented Generation (RAG) Chatbot will generate responses grounded in the files that are ingested, including confidence scores, to aide business analysis efforts",Implementation and Assessment,Neither,10/1/2023,3/15/2024,10/31/2024,,Developed with contracting resources.,TIRNO-17-D-00004 WR2730,No,,,No,,No,Yes,No,,"No models are being trained. Outputs are being calibrated through prompt engineering. The inputs will be supporting documentation available to the public in PDF format, as well as a subset of ALC code (.ASM files) approved for ingestion. No Personally Identifiable Information (PII) data will be used. Any additional inputs used will go through appropriate review process. ASM code is a shorthand for machine language. Each ASM language is designed for a specific computer architecture, and sometimes an operating system.","Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",None,,Yes,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Infrastructure is provided through innovation lab managed service.,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Modernization Accelerator - International Business Machines (IBM) Cognitive Data Mapper (CDM) and AWS SageMaker+LLAMA,Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,None of the above.,"The Business Masterfile Modernization (BMF Mod) project will reproduce the legacy BMF system, written in 60-year-old mainframe technology, into a modern, cloud-based system. BMF Mod will also add additional features to the system as new technology permits. Note that this software will only be used in an isolated development (sandbox) environment, to assist developers in programming the new modernized system. It will not be promoted into Production.  These Artificial Intelligence/Machine Language (AI/ML) tools analyze the existing legacy BMF system code, after it has been prepared by other non-AI/ML International Business Machines (IBM) tools, to automatically produce: Business logic in natural language, which describe what actions or data transformations the code performs: Modernized Java code to implement the business rules; Microservice recommendations, meaning that code snippets are grouped together by function and access patterns, to split very large programs that are hard to understand into smaller chunks that can be more easily maintained. These tools take in code from the existing BMF system, after it has been prepared by other non-AI/ML IBM tools, and analyze the code to produce the business logic, code, and microservice recommendations. IBM Cognitive Data Mapper (CDM) uses Natural Language Processing (NLP) Machine Learning (ML) models to automatically analyze the source system's data and metadata, map to a target model or business glossary, and/or classify the information. CDM will be used for automated data mapping to automatically identify and establish relationships between data elements in different data sources and source systems. Amazon Web Services (AWS) SageMaker with LLAMA accepts prepared code from other non-AI/ML IBM tools, then further analyzes the code to extract business logic and convert the code to modern Java. The tools analyze existing code from the BMF system, which are very large programs, and suggest how to translate the code into smaller, more manageable and understandable business logic, code, and microservices. All output business logic, code, and microservice recommendations will be reviewed by both IBM and IRS employees prior to being used in development of the new modernized BMF system. The outputs will be put into a spreadsheet that can be shared with subject matter experts, who will then manually check accuracy of the work. Code will also be packaged and extensively tested to ensure accuracy. These AI/ML tools will accelerate our development process, allowing us to deliver a modernized system a few years faster than otherwise. The output business logic and microservice recommendations are also very useful as documentation artifacts, helping to understand how the BMF system functions. Approval will allow BMF Mod work to continue. Without approval we would need to extend our modernization plans to account for performing these functions manually. While these AI/ML tools are only intended to be used by IBM consultants working on the BMF Mod program, its impacts will benefit all BMF stakeholders including IRS business operating divisions, Treasury, and taxpayers.","IBM CDM uses Natural Language Processing (NLP) Machine Learning (ML) models to automatically analyze the source system's data and metadata, map to a target model or business glossary, and/or classify the information. CDM will be used for automated data mapping to automatically identify and establish relationships between data elements in different data sources and source systems. AWS SageMaker with LLAMA accepts prepared code from other non-AI/ML IBM tools, then further analyzes the code to extract business logic and convert the code to modern Java.",Acquisition and/or Development,Neither,6/6/2024,6/14/2024,,,Developed with contracting resources.,2032H5-24-F-00181,No,,,,,,,Yes,,Existing legacy Business Master File code,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups.",None,,,,,No,,Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, Joint Committee on Taxation Research Aide,Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,None of the above.,The Joint Committee on Taxation Review (JCTR) Research Aide is a generative AI (GenAI) system that uses Retrieval Augmented Generation (RAG) search to answer questions on JCTR policies and procedures that are provided in natural language. This tool has a use case specific user interface (UI) that uses the Certara platform via Application Programming Interface (API) to perform the RAG search and generation.,"This system uses an embedding model to produce a vector representation of texts for contextual searching, and Large Language Model to generate a response to user inquiries. Outputs of the tool include the model generated response to a user's inquiry, as well as the text from relevant documents identified in the retrieval stage and used by the model in generating said response.",Acquisition and/or Development,Neither,1/15/2024,1/15/2024,,,Developed with contracting resources.,2032H8-23-F-00166,No,,,,,,,Yes,,Text from the sections of the Internal Revenue Manual relevant to the Joint Committee (JC) (4.36) and instructional/training documents provided by the IRS to JC examiners and staff.,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups.",None,,,,,Yes,RAS-1,Less than 6 months,Yes,,Other,We do not train any models associated with this use case. The LLMs are downloaded pretrained and hosted in the Certara tools.,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, Technical Demo of Large Language Models for Code Development,Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,None of the above.,"Problem: IRS developers spend significant time developing code and unit tests. IRS developers do not currently benefit from AI-assisted code development, which can greatly improve the accuracy and efficiency of development work. Use Case: Using llamafile and collama IDE plugins within VS Code, developers will be able to utilize code completion and suggestion, code generation, code refactoring, code documentation, debugging and unit testing capabilities to optimize the developer experience and produce high-integrity code solutions. Why now? Developers need some interim solution to leverage Generative AI (GenAI) capability while the long-term Enterprise Solution is being developed.","Text Generation, Summarization, Code Completion, Unit Tests, Text Translation, Text-to-Text Transformation.",Initiated,Neither,5/12/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Internal Revenue Manual Research Aid (IRMA),Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,None of the above.,IRMA - Internal Revenue Manual (IRM) Research Aide is a generative AI (GenAI) system that uses Retrieval Augmented Generation (RAG) search to answer questions on IRM policies and procedures that are provided in natural language. This tool has a use case specific user interface (UI) that uses the Certara platform via Application Programming Interface (API) to perform the RAG search and generation.,"This system uses an embedding model to produce a vector representation of texts for contextual searching, and Large Language Model to generate a response to user inquiries. Outputs of the tool include the model generated response to a user's inquiry, as well as the text from relevant documents identified in the retrieval stage and used by the model in generating said response.",Acquisition and/or Development,Neither,5/9/2024,5/10/2024,,,Developed with contracting resources.,2032H8-23-F-00166,No,,,,,,,Yes,,All text from the sections and subsections of the unredacted Internal Revenue Manual.,"Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",None,,,,,Yes,RAS-1,Less than 6 months,Yes,,Other,We do not train any models associated with this use case. The LLMs are downloaded pretrained and hosted in the Certara tools.,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, Tax Payer Notice Modernization,Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,Improving the quality of written communications using AI tools.,"The purpose of this AI project is to modernize and simplify tax notices for better taxpayer comprehension. The AI accomplishes this by transforming complex tax language into clear, accessible content. The model automates the generation of these revised notices, with human reviewers ensuring accuracy and relevance before dissemination. Anticipated benefits include reduced errors and inquiries, increased taxpayer satisfaction, and improved efficiency in tax administration. The direct impact is clearer communication with taxpayers, who are the primary population affected by this initiative, resulting in easier navigation of their tax obligations. ","Rewritten taxpayer notices presented in clear, concise language.",Acquisition and/or Development,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Enable Face ID to unlock M365 apps on Government iPhone and iPad,Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,Unlocking smartphones or other devices without the need for passwords or PINs using AI-based facial recognition.,Improve Ease of use for customer accessing Org Data in M365 apps and TEAMS on government iPhone and IPad.Customers would no longer need to continuously input Password into the M365 apps if Face ID is enabled. https://support.apple.com/en-us/102381,"The AI model unlocks government iPhones or iPads and Applications like Microsoft's Teams, Outlook and Other apps by comparing the customers face geometry. “Face ID analyzes thousands of invisible infrared dots to create a depth map of your face, which then captures an infrared image of your face.”  ",Implementation and Assessment,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Data Integration using Informatica Data Management Cloud,Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,None of the above.,"Informatica Intelligent Cloud Services Overview Informatica Intelligent Cloud Services (IICS) is a cloud-native data integration and management platform that enables organizations to connect, access, transform, and govern data across hybrid and multi-cloud environments. IICS leverages artificial intelligence and machine learning to automate and optimize data processes, enhance data quality, and accelerate data insights. IICS offers a range of services and features, such as: Data Integration: A service that enables users to design, develop, and deploy data integration workflows and tasks across cloud and on-premises sources and targets, using a graphical user interface or a code-based environment. Data Integration supports various data integration patterns, such as batch, real-time, streaming, and change data capture. The solution enables Integrated Production Model (IPM) to load data directly to the cloud from on-premises sources. Without the solution, there's no native support to load data to Databricks, causing extra work and less-than-optimal solutions to maintain and develop. Any custom needed IPM data is impacted by this solution.","The Informatica Artificial Intelligence (AI), named Claire, can make the following types of recommendations during mapping design: - Recommend transformation types to include in the mapping. - Recommend additional sources based on primary key and foreign key relationships. - Recommend joining or unioning additional sources. - Make recommendations for objects in a mapping inventory. - Recommend mapplets or user-defined functions. - Make recommendations to standardize your data - Make recommendations to mask sensitive data. - Recommend rule specifications.",Acquisition and/or Development,Neither,4/1/2024,11/1/2023,,,Developed with contracting resources.,2032H5-21-F-00059 / NNG15SD22B,No,,,,,,,Yes,,The source information comes from the Informatica mapping development work.,"Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",None,,,,,No,,More than 12 months,Yes,,Yes,,Yes,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, Answering Employee Questions with Natural Language Processing in IRWorks,Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,None of the above.,"The purpose of the Artificial Intelligence (AI) element of the IRWorks platform (a ServiceNow implementation for the IRS) is to utilize natural language to respond to common user questions about IRWorks knowledge base (KB) articles and service catalogue items in the IRWorks Employee Center. This AI element is interpretive. At a high level, this is accomplished by using language models to train the Virtual Agent (a ServiceNow product) to respond to user questions. The decision/judgement that this AI element uses is to search existing Knowledge Base (KBs)/service catalogue items, in conjunction with the user's defined access within IRWorks, to automate Tier 1 support by outputting (in natural language) answers and/or associated links that align to the customer's (natural language) query. Yes, humans are involved in reviewing the output - the AI element does not produce actionable items but produces information or links that are provided to the querying user for them to review and/or take manual actions. The anticipated benefits of this AI element of the IRWorks platform are to reduce costs of Tier 1 support and provide a better user experience. The direct/ultimate impact of this decision is to reduce costs of Tier 1 support and provide a better user experience. The population impacted by this decision is all IRWorks users, about 90K+ IRS employees.",Improve efficiency and optimize outputs for end users requesting services/information from within ServiceNow.,Operation and Maintenance,Neither,4/30/2023,4/30/2023,4/30/2023,,Developed with contracting resources.,47QTCA22D00B2; GS35F0414R; 47QTCA22D00B2,No,,,No,,No,Yes,Yes,,IRS IRWorks Knowledge Management,"Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",None,,No,"Yes – agency has access to source code, but it is not public.",,Yes,IRWorks,Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Compliance Data Warehouse Data Access Log Monitoring,Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,None of the above.,"The purpose of this AI project is to provide insights into access and usage patterns improving understanding of Compliance Data Warehouse (CDW) data usage and Personally Identifiable Information (PII) exposure risks. It does so by comparing a user's activity to a distribution of similar users, based on access groups, and provides a score based on how far from expected normal behavior that user's activity performed. This model is only used to inform humans rather than automating a decision. This model automates the manual analysis of log data to identify and rank anomalous user activity for further investigation. Humans review the AI output prior to taking action on the result. They do so by reviewing AI outputs (i.e., model scores, feature ranking, user's historic log data) in a Tableau Dashboard and automated emailed reports. The benefits of this project are the automated multivariate analysis of CDW usage data in order to flag anomalous user activity that may otherwise go unnoticed via manual univariate analysis. The direct impact of this decision is the ability to routinely identify and evaluate high-risk events to avoid future security risks. No population is directly impacted by this AI as it is only used to inform the RAAS Infrastructure Security Team in order to conduct further outreach and investigation of anomalous user activity.","The AI model outputs a table of scored query activity per user per day where a higher score is interpreted as more anomalous activity, as well as feature ranking and score for each given output row where the score of a feature is the distance from the normal value for the population for that feature. The output of the broader system also includes supplemental score data such as flagging reoccurring anomalous behavior and the user's activity score percentile compared to the group they were modeled against. The model scores in addition to contextual information (i.e., discovery directory information and entitlement information) and supplemental statistics are displayed in a dashboard for the intended users.",Acquisition and/or Development,Neither,2/8/2024,2/8/2024,,,Developed with contracting resources.,2032H5-22-F-00432,No,,,,,,,Other,Our team produces an aggregated set of metrics from the raw CDW production log data.,"Our team produces an aggregated set of metrics from the raw CDW production log data. This is merged with internally available employee information for identifying users by business unit, and entitlement information to identify Common and Special Access Groups (CAGs/SAGs).","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups.",None,,,,,Yes,RAS-1,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, Volunteer Income Tax Assistance (VITA) Generative AI Chatbot Proof of Concept,Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,Searching for information using AI.,"The objective of this AI project is to investigate the feasibility and applicability of using Large Learning Model, Retrieval-Augmented Generation AI technology to consolidate multiple sources of designated IRS publicly facing information into a simple question and answer tool. It accomplishes this purpose by using a select data set to retrieve answers for questions asked by a small subset of Volunteer Income Tax Assistance (VITA) personnel during tax preparation (this is the Proof of Concept's use case). Currently, volunteers have to know where to go to find the information for which they are looking, across multiple, detailed documents. This includes publications, forms, certification training guides, etc. Once they determine which document to use, then they have to search for answer within that document. This model uses the information provided (designated, publicly available IRS publications) to search for an answer and shows the user what document the information came from as well as the page. This saves the user a lot of time and effort. This model has been tested by two separate groups with a 99.5% accuracy rate for answers to questions asked. Humans were very involved in the development and testing of this chatbot to ensure answers were correct, accurate and relevant to the questions asked. The chatbot was built with a survey-type feature whereby each answer is scored by the volunteer for the previously mentioned criteria. The questions, answers, scores, and comments (feedback) from the volunteers is recorded within the chatbot for review and analysis to make further adjustments or fine tuning of the chatbot. Chatbot input that resembles Personally Identifiable Information (PII) is blocked, so users cannot ask questions that include any form of PII. The benefits of this proof-of-concept chatbot and this specific technology are to decrease the time and effort to prepare taxes (by using an IRS reference tool) and the volunteers will be able to provide more efficient service for taxpayers. During initial Subject Matter Expert (SME) user testing, it was discovered some volunteers are already using publicly available Generative AI (GenAI) tools to find information. This information is coming from all sources in the web-world, or wherever that GenAI tool is directed for information gathering. The free Google Gemini tool is one of the examples a SME tester referenced. If the IRS VITA GenAI chatbot tool is using information the IRS provides as the ONLY data source it uses as reference to provide an answer, the volunteer will know it is the correct answer, instead of the volunteer using a tool of their choice and assuming the answer is correct. This is a proof of concept and will expire at the end of the upcoming tax season (April 2025), The population impacted is a small number of VITA volunteers who have already been Certified under the VITA program and who Taxpayer Services and who agree to participate in the proof of concept to help determine its usefulness and applicability. Stakeholder Partnerships, Education and Communication has vetted and asked to test while they are preparing taxes for those eligible under the VITA/Tax Counseling for the Elderly (TCE) Programs. The number will not exceed 30 testers.",chatbot-generated answers (to user-entered prompts; aka questions),Acquisition and/or Development,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Ask-CFO Research Aide,Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,None of the above.,"Ask-CFO is an extension of the Research Aide developed using the Certara Composer platform originally designed for the Joint Committee on Taxation Research Aide (JCR) and Internal Revenue Manual Research Aid (IRMA) use cases. It is a Retrieval Augmented Generation (RAG) search on a number of Chief Financial Officer (CFO) specific documents. In addition to the existing research aide capability and user interface (UI), extensions will be developed to incorporate AI Agents that can perform actions such as document summary comparison over fiscal year, or extract information from tables.","This system uses an embedding model to produce a vector representation of texts for contextual searching, and Large Language Model to generate a response to user inquiries. Outputs of the tool include the model generated response to a user's inquiry, as well as the text from relevant documents identified in the retrieval stage and used by the model in generating said response.",Acquisition and/or Development,Neither,8/1/2024,8/1/2024,,,Developed with contracting resources.,2032H823F00166,No,,,,,,,No,,Our team was provided with a sample of documents and reports used and produced by the office of the CFO. Text from these documents is used in this use-case.,"Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",None,,,,,Yes,RAS-1,Less than 6 months,Yes,,Other,We do not train any models associated with this use case. The LLMs are downloaded pretrained and hosted in the Certara tools.,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, Redaction Studio - Redaction Assistant,Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,None of the above.,"Redaction Studio is a Commercial Off The Shelf (COTS) application that allows Criminal Investigation (CI) Special Agents to efficiently and securely redact body worn camera footage. The application provides an ""AI masking"" feature called Redaction Assistant that will attempt to automatically identify and mask (i.e. redact) items like faces. The masks proposed by the feature can be reviewed, adjusted, or removed by the Special Agent using the application. IRS:CI is not training or tuning a model, we are just proposing that Agents be able to use an ""AI""/ML feature. Efficient redaction of body worn camera footage is needed to comply with Executive Order 14047 which requires the expedited public release of body worn camera footage following incidents involving serious bodily injury or death. Axon, the creator of Redaction Studio, describes the assistant like this: ""The Redaction Assistant is an advanced AI tool within Redaction Studio that significantly speeds up the redaction process. By scanning through footage, it finds and suggests areas that may need redaction, such as faces or personal information, saving time and enhancing accuracy. Redaction Assistant is an Axon Redaction studio add-on and might not be available in all agencies."" Some details on using it can be reviewed here: https://my.axon.com/s/article/Using-Redaction-Assistant?language=en_US","Axon, the creator of Redaction Studio describes the assistant like this: ""The Redaction Assistant is an advanced AI tool within Redaction Studio that significantly speeds up the redaction process. By scanning through footage, it finds and suggests areas that may need redaction, such as faces or personal information, saving time and enhancing accuracy.""",Implementation and Assessment,Neither,1/1/2023,7/16/2024,11/30/2024,,Developed with contracting resources.,2032H5-24-D-00007,No,,,No,,No,Yes,Other,This use case does not require datasets,"This use case does not require datasets and does not involve training, fine-tuning or evaluation of a model.","Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,No,No – agency does not have access to source code.,,Yes,"CI-1 ,CI-2",Less than 6 months,Other,No infrastructure required,Other,"No process required, no training, no fine-tuning.",Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Intelligent Document Management System (IDMS),Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,Searching for information using AI.,"IDMS is a vendor managed cloud platform solution which will provide an enterprise capability to process, store, retrieve, manage, and share a variety of documents. Utilizing a Platform-as-a-Service (PaaS) model, this robust cloud-hosted platform will improve the overall document management user experience, utilizing artificial intelligence and machine learning to automate extraction, processing, and analyzing of critical data from multiple structured and unstructured formats. In this way, IDMS will minimize the need for manual document processing, scaling to rapidly process large volumes of data accurately, reducing data processing costs and swiftly delivering responses with detailed and relevant information. This allows the customer to review the AI output to enhance their decision-making process. The initial customer is the Office of Chief Counsel. IDMS will integrate with existing and future critical business platforms to provide a seamless end-to-end user experience.",Output of the model is usually a JavaScript Object Notation (JSON) text payload resulting from the Inference Application Programming Interface (API) calls with unique id referencing the document source,Initiated,"Rights-Impacting ",,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Taxpayer 90 (Palantir prototype),Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,None of the above.,"Palantir AIP will interact with synthetic data and public information (e.g. IRM and tax forms) to summarize documentation and parse through information more quickly. AIP is enabled within SaaS platform (only within SaaS, no firewall changes/proxy exceptions needed), shared with Selection aNd Analytic Platform (SNAP), but AIP is only permissioned to users with the Taxpayer 90 Business Entitlement Access Request System (BEARS) entitlement. Data is contained within the Software as a Service (SaaS) tenant.",Summary of synthetic data and public information (e.g. Internal Revenue Manual (IRM) and tax forms).,Acquisition and/or Development,Neither,10/30/2024,9/4/2024,,,Developed with contracting resources.,2032H5-18-A-00029,No,,,,,,,No,,Test data provided only for prototype development,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,,,,Yes,SNAP-TP90,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, Expanded Systran Translation Software,Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,None of the above.,"Internal Revenue Service-Criminal Investigation (IRS-CI) is looking to acquire a high-volume enterprise license for Systran. This is the same product that Research, Applied Analytics and Statistics (RAAS) currently uses, but will be expanded to allow high volume processing through multiple Graphic Processing Units (GPUs), enabling large quantities of documents to be translated in a relatively short time. IRS-CI needs this capability to translate seized data in foreign languages. ",The output is a PDF document containing the translation of the documents.,Acquisition and/or Development,Neither,3/15/2024,11/5/2024,,,Developed with contracting resources.,No contract number as acquisition is still pending,No,,,,,,,Other,Not needed. ,None,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,,,,Yes,"CI-1, CI-2",Less than 6 months,Yes,,Other,not applicable. ,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,"Secure storage of content at rest and in flight, we are using Google Cloud resource that will be hardened and secured in the IRS environment for production deployment",,,,,,,,,,,,,, Transcription -- Evidence.com,Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,Transcribing and summarizing a recorded meeting or interview using AI.,Evidence.com provides an accurate AI based transcription tool for law enforcement recordings.,The evidence.com site produces a word document.,Implementation and Assessment,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Digital Forensics Chatbot for IRS Employees,Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,Searching for information using AI.,"a chatbot style question/answer application for all things digital forensics. This could help examiners with new and emerging technology when encountered in the field, help write bash and python scripts to manage data, basically anything digital forensics. Currently examiners have to rely on google and other open source information to get answers to questions that arise.","Output would include information as to specific hex or code sequences, location of stored data on various system, information as to the access or known exploits of encountered systems to name a few. ",Acquisition and/or Development,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Generative AI for Form 1040X: Tax Examiner Assistant,Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,None of the above.,"Develop a workflow assistant that leverages generative AI to consolidate and analyze complex policies from various documents (e.g., IRMs, SERP, desk guides). This tool will leverage Large Language Model (LLM) to provide real-time, accurate guidance for Tax Examiners handling individual amended returns. Future iterations will incorporate additional features that may include generating correspondence for taxpayers that prioritizes their actions. Key stakeholders, including Research, Applied Analytics, and Statistics (RAAS), Information Technology (IT), Submission Processing, and Accounts Management, will collaborate to enhance existing RAAS capabilities, integrate new documents, and fine-tune the solution for this specific use case. The tool will be made available to a select pool of Tax Examiners, who will verify and validate the outputs of the tool prior to finalizing the processing of the amended returns. Benefits and impacts include operational efficiency, reduce processing times, and provide timely, accurate guidance.",The 1040X Tax Examiner Assistant tool produces a comprehensive response with a plain text summary of the answer with relevant resources and links to the resources. The tool also outputs a feedback prompt where Tax Examiners can provide user feedback.,Acquisition and/or Development,Neither,9/3/2024,9/12/2024,,,Developed with contracting resources.,2032H5-24-P-00086,No,,,,,,,No,,"Text and excerpts from publicly available, non-sensitive data from Internal Revenue Manual (IRM) 21, IRM 3, and Document 6209.","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,,,,No,,Less than 6 months,No,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, ServiceNow Generative AI Pilot,Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,Summarizing the key points of a lengthy report using AI.,"As part of a Chief Technology Officer (CTO) sponsored objective to enable and utilize Generative Artificial Intelligence (Gen AI) capabilities the User and Network Services Information Technology (IT) Service Desk in cooperation with Enterprise Operations will be implementing a limited production challenge to support AI generated Incident Summarization; providing agents with concise summary of case notes and history, Resolution Notes generation; generating accurate resolution notes based on actions taken and solution achieved, and Knowledge Article Generation; generating complete knowledgebase articles base on incident or case records. With the above we expect to see significant time savings when conducting warm handoffs between teams, while reducing the mean time to restore incidents. Additionally, the effort should ensure high-quality resolution notes are generated to assist in solving future incidents, while feeding existing knowledge review and publication processes potentially empowering self-service capabilities which will reduce time to closure activities.","Incident Summarization; providing agents with concise summary of case notes and history, Resolution Notes generation; generating accurate resolution notes based on actions taken and solution achieved, and Knowledge Article Generation; generating complete knowledgebase articles base on incident or case records",Initiated,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Circular Dependencies (Individual Master File Modernization),Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,None of the above.,"Purpose/Objective of the AI Project: To accelerate resolution of circular dependencies (cases where calls to classes create dependencies upon themselves Class A --> Class B --> Class C --> Class A) using Large Language Model (LLM) How It Accomplishes Its Purpose: Classes involved in circular dependencies are provided to the LLM and resolutions are proposed for the end user to accept Decision or Judgment Automated by the Model: Resolution of circular dependencies identified in Individual Tax Processing Engine (ITPE) codebase Human Review of Artificial Intelligence (AI) Output: Yes, human review is required before accepting any changes Anticipated Benefits: Proposed resolutions for circular dependencies which are projected to accelerate the process from months to hours Direct or Ultimate Impact: Accelerating resolution of circular dependencies in ITPE Population Impacted by This Decision: ITPE Architects and Developers",Recommendations on resolutions of circular dependencies identified in ITPE.,Initiated,Neither,4/11/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Synthetic Test Data Generation (Individual Master File Modernization),Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,None of the above.,"Purpose/Objective of the Artificial Intelligence (AI) Project: The primary objective of this AI project is to create rich and voluminous synthetic data for testing purposes. This synthetic data will be instrumental in enhancing the quality and comprehensiveness of various testing processes. How It Accomplishes Its Purpose: At a high level, the project achieves its goal by ingesting data model schemas and generating scripts. These scripts are then used to create synthetic test data, ensuring that the generated data closely mimics real-world scenarios and datasets. Decision or Judgment Automated by the Model: This AI tool automates the creation of synthetic data for testing purposes. It systematically generates data that can be used in various testing scenarios without the need for manual data creation. Human Review of AI Output: Yes, humans are involved in reviewing the AI-generated synthetic data prior to taking action on the results. This step ensures that the data meets the required standards and is suitable for the intended testing purposes. Anticipated Benefits: The project is expected to provide a robust framework for generating synthetic test data. This framework will facilitate the creation of comprehensive and high-volume test data, which is essential for conducting thorough performance, regression, and other types of tests. Direct or Ultimate Impact: The ultimate impact of this decision is the availability of more comprehensive and increased volumes of test data. This enhanced data will support various testing activities, leading to more reliable and effective testing outcomes. Population Impacted by This Decision: The primary population impacted by this decision includes testers and end users across ITPE and Individual Master File (IMF). These stakeholders will benefit from improved testing processes and outcomes.",Synthetic Test Data customized to user specification / requirements.,Initiated,Neither,10/21/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Auto Tech Debt Remedy (Individual Master File Modernization),Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,None of the above.,"Purpose/Objective: The purpose of this Artificial Intelligence (AI) project is to utilize industry-standard scanning tools and generative AI capabilities to automate the resolution of technical debt in source code. How It Accomplishes Its Purpose: The project identifies technical debt using industry-standard scanning tools such as SonarQube. The output from these tools is then fed into a large language model, which generates the modified, resolved code. This code is integrated into an automated pipeline to streamline the entire process. Decision or Judgment Automated: The model automates the writing of code to resolve and remediate technical debt. Human Review Process: Yes, humans are involved in reviewing the AI output prior to taking action. Specifically, a pull request (PR) is created in GitHub through this tool, and no code is pushed without human review. Anticipated Benefits: The primary benefit anticipated from this project is the reduction of technical debt. Direct or Ultimate Impact: The direct impact is reduced technical debt, leading to more maintainable and efficient codebases. Population Impacted: The immediate population impacted by this decision includes software developers. In the long run, end users will also benefit as the software will be maintained more easily and efficiently.",Large Language Model (LLM) recommendations for code remediations identified from application scanning tools.,Initiated,Neither,10/21/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, M365 Copilot,Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,Improving the quality of written communications using AI tools.,"An established group of participants across multiple business areas will test Artificial Intelligence-based features and functions available within the existing M365 applications using M365 Copilot. The testing results will validate the security, benefits and capabilities that support the existing architectural boundaries of M365. ",User defined prompts,Initiated,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Copilot for Power Platform,Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,None of the above.,"An established small group of participants across multiple business areas will test Artificial Intelligence-based features and functions available within the existing Power Platform applications using Copilot. The testing results will validate the security, benefits and capabilities that support the existing architectural boundaries of Power Platform. ",N/A. This is an Artificial Intelligence assistant that will support development activities.,Initiated,Neither,11/13/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Copilot for M365 (Criminal Investigation Unit),Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,Improving the quality of written communications using AI tools.,"1 **Word**: Copilot can help draft, edit, and summarize documents, providing suggestions for grammar, style, and tone improvements. 2. **Excel**: It can analyze data, generate charts, and create complex formulas, making data management more efficient. 3. **PowerPoint**: Copilot assists in designing presentations, suggesting slide layouts, and generating content for slides. 4. **SharePoint**: It enhances collaboration by organizing and managing documents, automating workflows, and providing insights into document usage. 5. **Outlook**: Copilot helps manage emails by drafting responses, summarizing long email threads, and organizing your inbox. 6. **Teams**: It facilitates meeting preparation by generating agendas, taking notes, and providing meeting recaps.","1 **Word**: Copilot can help draft, edit, and summarize documents, providing suggestions for grammar, style, and tone improvements. 2. **Excel**: It can analyze data, generate charts, and create complex formulas, making data management more efficient. 3. **PowerPoint**: Copilot assists in designing presentations, suggesting slide layouts, and generating content for slides. 4. **SharePoint**: It enhances collaboration by organizing and managing documents, automating workflows, and providing insights into document usage. 5. **Outlook**: Copilot helps manage emails by drafting responses, summarizing long email threads, and organizing your inbox. 6. **Teams**: It facilitates meeting preparation by generating agendas, taking notes, and providing meeting recaps.",Acquisition and/or Development,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Automated Collection System (ACS) Voicebot,Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,None of the above.,"This project developed self-service application in four phases that allows taxpayer to mitigate various issues regarding payments, balance dues, account history etc. SBSE Phase 1-4: Automated Collection System (ACS) Conversational Interactive Voice Response (IVR) (ACI) Phase 1, Voice Balance Due (VDB): Taxpayers may set up payment plans, extensions, monthly payment amounts, payment dates, revise agreements and form of payment. (Prod Date: 6/14/22) ACI Phase 2, Location and Transcript: Provides taxpayers with information regarding mailing addresses for forms based off the caller’s zip code. Transcript conducts enhanced business operating division routing (EBR) checks and enables taxpayers to obtain a transcript of their tax return, and/or transcript of their account. (Prod Date: 8/23/22) ACI Phase 3, Payoff and View Credit: Allows taxpayers to find out the amount of any outstanding balances with the IRS. This includes Get Payoff Date, Mailing Instructions and the ability to transfer to VBD. (Prod Date: 9/28/22) ACI Phase 4, View Debit: Authorizes and allows the taxpayer to choose any tax year on record and the application will provide a history of their account. (Prod Date: 1/5/23) The Intent Engine (using NLP and AI algorithms) classifies taxpayers' utterances (questions) into intents (requests). The responses of the voicebot and business logic is predetermined by content owners. This AI tool aims to classify/navigate to a correct predetermined response.","The AI outputs include responses tailored to taxpayer queries, such as payment plan setup, balance inquires, mailing address information, transcript requests, and account history details. If the AI cannot classify the input confidently, it prompts for clarification or escalates to a live agent.",Operation and Maintenance,Neither,5/3/2021,1/18/2022,6/14/2022,,Developed with contracting resources.,2032H5-19-F-00641,No,,,No,,No,Yes,Yes,,"System uses historical data from ACS call logs, payments plans, and account balance queries. ","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups.",None,,No,No – agency does not have access to source code.,,Yes,Nuance Digital Contact Center Platform (DCCP),Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, Customer Experience Analytics,Department of the Treasury,TREAS,Internal Revenue Service,Mission-Enabling,,None of the above.,Extract insights from data sources in the customer service domain to better understand service interactions and create new opportunities to improve the customer experience.,Summary of call transcripts between IRS and Taxpayer,Acquisition and/or Development,Neither,10/1/2023,10/2/2023,,,Developed with contracting resources.,2032H8-21-F-00179,No,,,,,,,Yes,,Call Transcripts,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,,,,Yes,RAS-1,Less than 6 months,No,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, Exercise Design,Department of the Treasury,TREAS,DF-OCCIP,Mission-Enabling,,Summarizing the key points of a lengthy report using AI.,"Expedite the exercise development process through more rapid event design and scenario creation as well as streamline post-event reporting. Access to a Large Language Model (LLM) can significantly enhance an exercise program primarily in two ways. First, by improving the development of scenarios and second, post-event analysis. For scenario development, an LLM can quickly generate realistic and complex scenarios tailored to specific objectives and characteristics. For example, in a cybersecurity exercise, the LLM can create detailed narratives involving various types of cyber incidents, such as ransomware attacks, data breaches, and phishing schemes. It can simulate the actions of adversaries, the impact on IT infrastructure, and the necessary response measures. This variety ensures participants are exposed to a wide range of potential cyber threats, enhancing their preparedness and response capabilities. ","Detailed event descriptions, scenario information, and/or reporting and analysis.",Operation and Maintenance,Neither,10/8/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Incident Coordination and Reporting Support,Department of the Treasury,TREAS,DF-OCCIP,Mission-Enabling,,Summarizing the key points of a lengthy report using AI.,"Reduce development time for products by rapidly summarizing latest information and generating a draft product for human review. Access to a LLM could significantly reduce the time needed for Incident Coordinators to develop (and edit) the first version of incident related products. Inputs from public and potentially non-public sources could quickly be ingested by a model and refined into the products based on desired templates and characterizations. Additionally, if the model uses live data, it could be used for research and gathering updates as the incident progresses. The benefits of having access to a LLM are particularly significant at the outset of an incident when all members of the incident response function are trying to manage multiple inputs from a variety of internal and external sources while attempting to generate products for socialization with key partners. ",Quick turn reports based on existing templates for sharing with partners ,Operation and Maintenance,Neither,10/8/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, EFTPS Fraud Monitoring,Department of the Treasury,TREAS,Bureau of the Fiscal Service,Mission-Enabling,,Searching for information using AI.,"The Electronic Federal Tax Payment System (EFTPS) Fraud Initiative performs fraud detection within the Direct Pay and EFTPS Online payment channels. It supports near-real-time identification, profiling, scoring, and alerting of suspicious transactions and behavior. When user activity is captured, the model compares features of the activity/payment to EFTPS historical data and produces a score based on assessed risk level. This risk score generates an alert in the solution case manager for further review by a fraud analyst. The results of the [human] review could warrant blocking of the user account, monitoring of future activity, or escalation to law enforcement for further investigation.",Risk Score for Potential Fraud Prevention,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Digitization of Microfilm,Department of the Treasury,TREAS,Bureau of the Fiscal Service,Mission-Enabling,,Digitizing text from scanned documents or smart forms for archival purposes using AI.,"In support of the microfilm conversion process for Mature Undredeemed Debt/SavingsBonds, the Fiscal Vendor, Iron Mountain, has incorporated a focus on the application of AI and deep learning ML to maximize processing efficiency and accuracy of results. The solution leverages these technologies to continually optimize output and ultimately our return on investment. Iron Mountain's solution uses AI/ML to make data more accessible and enabled Fiscal Service to establish the abilit to search the data and identify bond owners. To date, 175,000 reels of film have been digitized for approximately 2.1B images that are ultimately stored in Azure cloud tenant.",Image Files,Implementation and Assessment,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Check Fraud Pipeline,Department of the Treasury,TREAS,Bureau of the Fiscal Service,Mission-Enabling,,Searching for information using AI.,"Since the dramatic spike in Economic Impact Payments and Advance Child Tax Credit checks, there has been a sharp uptick in U.S. Treasury check fraud. Specifically, fraudsters have been stealing U.S. Treasury checks and altering the names on the checks. The situation is not typically detected until a payee claimed non-receipt to issuing agency (often long after fraud occurred, also after time limits to recover from FI). Additionally, Law Enforcement has been challenged in recovering evidence (e.g., ATM footage) to prosecute, as well as identifying organized fraud at the transactional level. This pilot has enabled the streaming of check images into the payment analytics environment and is leveraging AI/ML to read check image and check settlement data for comparison against check issue data. To date, 30 million check images and settlement transactions have been analyzed identifying 12K fraudulent altered items worth over $100M",Risk Score for Potential Fraudulent Checks,Implementation and Assessment,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Gender differentiated credit scoring,United States Agency for International Development,USAID,"USAID/Bureau for Inclusive Growth, Partnerships, and Innovation",Diplomacy & Trade,,None of the above.,"University of California, Berkeley, is building a machine learning model to conduct gender differentiated credit scoring for customers of Rappicard in Mexico. They will compare this ML model to Rappi's ""status quo"" model to determine whether a gender differentiated model leads to greater access to credit for women.",Credit scores,Retired,Neither,7/14/1905,7/14/1905,7/15/1905,7/15/1905,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Using ML for predicting treatment interruption among PLHIV in Nigeria,United States Agency for International Development,USAID,USAID/Bureau for Global Health (GH),Health & Medical,,None of the above.,"Using data from USAID funded Strengthening Integrated Delivery of HIV/AIDS Services (SIDHAS) project in Nigeria we trained and tested an algorithm that can be used for predicting the probability that someone newly initiated on ART will interrupt treatment. The algorithm has been successfully integrated into the Lafiya Management Information System (LAMIS), the individual-level client level electronic medical record system. Each week the outputs, for each new patient is shared with staff at the health facilities and those at high risk are provided with more intensive follow up support to reduce the risk of treatment interruption. We also conducted a qualitative assessment among to health care workers at the facilities to determine their perception of ML and determine what additional support are required for institutionalizing ML into their routine work. ",Patient risk scores,Retired,Both,7/13/1905,7/13/1905,7/13/1905,7/15/1905,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Machine Learning for Peace,United States Agency for International Development,USAID,"USAID/Bureau for Democracy, Human Rights, and Governance",Diplomacy & Trade,,None of the above.,"DRG is partnering with a seven-member consortium, including researchers at Duke University’s DevLab, to build an Artificial Intelligence (AI) tool that can forecast closing civic spaces -- patterns of events that result in curtailed freedoms and impaired democracy.","Event labels, forecasts",Operation and Maintenance,Neither,7/10/1905,7/10/1905,7/14/1905,,Developed with contracting resources.,GS-10F-0294V/7200AA23M00014,No,,,No,,No,No,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,N/A,Yes,No – agency does not have access to source code.,,,,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Long-term impacts of land-use/land-cover dynamics on surface water quality in Botswana’s reservoirs using satellite data and artificial intelligence methods: Case study of the Botswana’s Limpopo River Basin (1984-2019),United States Agency for International Development,USAID,"USAID/Bureau for Inclusive Growth, Partnerships, and Innovation",Science & Space,,None of the above.,"For water supply, semi-arid Botswana relies on the reservoirs within the Botswana’s LRB. Reservoirs are particularly susceptible to the negative impacts of land-use and land-cover (LULC) activities and runoff because of their complex dynamics, relatively longer water residence times, and their role as an integrating sink for pollutants from their drainage basins. Despite these interrelationships and significance in regional and global economic stability, land and water (L-W) are often treated in “silos”. To understand the complex L-W nexus within the LRB, this study will use data-driven artificial intelligence for quantitative determination of the relationships between LULC change, together with socioeconomic development indicators and climate change, and their impacts on water quality and availability within the basin, both for 1984-2019 and to predict future scenarios (2020-2050). To advance data acquisition for LULC analysis and climate change, the study utilizes optical Earth-observation and meteorological satellite data. To provide near real-time and cost-effective approach for continuous monitoring of reservoir water quality within the basin, the study will develop empirical models for water quality estimation and water quality index mapping using 35-years of in-situ water quality measurements and water spectral observations using drone-borne spectrometer and optical satellite imagery through regression modeling and geospatial methods.","Reports, visualizations",Retired,Neither,7/13/1905,7/13/1905,7/13/1905,7/15/1905,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Morogoro youth empowerment through establishment of social innovation (YEESI) lab for problem-centered training in machine vision,United States Agency for International Development,USAID,"USAID/Bureau for Inclusive Growth, Partnerships, and Innovation",Diplomacy & Trade,,None of the above.,"The project established the YEESI social innovation lab at Sokoine University of Agriculture to become a center of excellence for advancing ICT education in Tanzania. Students at YEESI are trained in Machine Learning (ML), Machine Vision (MV), and Natural Language Processing (NLP) applications to address the most pressing agricultural challenges in the country. ",Over 650 students received training at YEESI during the project. The Lab established the first comprehensive open-source machine vision dataset for central regions of Tanzania and worked with agriculture start-ups and NGOs to improve sustainable agricultural practices and economic growth.,Retired,Neither,7/13/1905,7/13/1905,7/13/1905,3/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Breakthrough RESEARCH’s Social Media Listening,United States Agency for International Development,USAID,USAID/Bureau for Global Health (GH),Diplomacy & Trade,,None of the above.,"Social media listening draws on machine learning to synthesize and organize the vast quantities of data shared over social media platforms. Breakthrough RESEARCH carried out social listening on 12,301 social media posts in Nigeria to explore how gender-related online conversations manifest themselves and whether they have changed in the last five years. Using Crimson Hexagon’s machine learning algorithm, “Brightview,” publicly available social media content originating in the countries of interest was scraped by the algorithm, for posts relevant to RH/FP and youth. The resulting social media posts were then classified by topic, using language detected in the content. This provided a dataset categorizing conversations into overarching topics, allowing analyses to uncover key trends in topic specific conversation volume, insights about misinformation, attitudes and social norms, and more. The machine learning algorithm was able to identify relevant social media content. The 12,301 social media posts were qualitatively assessed and categorized, allowing researchers to monitor and track social media conversations far more expansively than allowed by research methods more traditionally used in public health and SBC programs.","Reports, visualizations",Retired,Neither,7/14/1905,7/14/1905,7/14/1905,7/15/1905,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Serbia: AI predictions for the utilization of hospital beds,United States Agency for International Development,USAID,USAID/Bureau for Global Health (GH),Health & Medical,,None of the above.,"AI technology was used to predict bed occupancy at hospitals with MoH data from 2019, with an overall median error by department around 20%. This was a proof-of-concept model developed at the request of the Institute of Public Health (IPH) Batut to understand how AI can work and the value add. CHISU was asked to subsequently focus on a different use case (waiting list optimization for scheduled imaging diagnostics services, specifically CT and MRI), which is considered higher priority to demonstrate the implementation of the national AI strategy and the effect of AI in data use for decision making by the government, and will be addressed in the 2023-4.",Time series,Retired,Neither,7/14/1905,7/14/1905,7/14/1905,7/15/1905,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Qure.AI integrated in mobile X-ray screening to improve community TB case finding,United States Agency for International Development,USAID,USAID/Vietnam,Health & Medical,,None of the above.,The USAID Support to End TB project applies Qure.AI in mobile X-ray van to improve the quality of X-ray screening under community active TB case finding efforts in Vietnam. AI system is also critical to shorten the screening time and improve TB detection accuracy.,Image classification,Operation and Maintenance,Both,7/12/1905,7/13/1905,7/13/1905,,Developed with contracting resources.,72044020CA00002,No,,,No,,No,No,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,No,No – agency does not have access to source code.,,No,,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,No,,,,,No - Some individual decisions or actions require direct human oversight.,,,,,,,,,,, AI for Predictive Analysis of patients on HIV treatment,United States Agency for International Development,USAID,USAID/India,Health & Medical,,None of the above.,JHU ACCELERATE used Artificial Intelligence (AI)/Machine Learning for identifying People living with HIV(PLHIV) on treatment who are at risk of becoming lost-to-follow-up (LFU). Identifying the PLHIV on treatment who are at risk of becoming LFU will allow the project team to provide pre-emptive counselling and intervention to target/customize approaches patients who could become LFU.,Patient risk scores,Retired,Both,,,,7/14/1905,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Private Sector Engagement Evidence Gap Map Analytical Pilot,United States Agency for International Development,USAID,"USAID/Bureau for Inclusive Growth, Partnerships, and Innovation",Mission-Enabling,,None of the above.,The Private Sector Engagement Evidence Gap Map (EGM) Analytical Pilot examined the viability of using natural language processing to facilitate analysis of the EGM documents at scale (https://crcresearch.github.io/usaid-pse-egm/#/egm). This use case should be regarded as exploratory in nature.,"Reports, visualizations",Retired,Neither,,,,7/14/1905,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Risk-based modeling of poacher movements in protected areas,United States Agency for International Development,USAID,USAID/Southern Africa,Diplomacy & Trade,,None of the above.,"The purpose of the risk-based modeling of poacher movements in protected areas AI use case is to predict the risk of poaching activity in a given location within a national park/ conservation area. This model is intended for use by protected area managers to allocate resources based on poaching risk. The model can present risk (outputs) based on 1) an inferred historical binary prediction (threat/ no threat); 2) a risk category or converted probability (low/ medium/ high); or 3) by predicting specific poaching activities near a given location (e.g., types of activities: footprints; snares; wildlife carcasses; etc.). At the current stage/ iteration, inputs include a variety of data that can be grouped into the following categories: animal tracking data; terrain/ habitat/ infrastructure data; park/ conservation area historical data; and spatial/ temporal data.",Annotated maps,Retired,Neither,,,,7/14/1905,,,,,,,,,,,,N/A,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, CARI/El Salvador: Risk factors at community level that cause Internal Displacement,United States Agency for International Development,USAID,USAID/Bureau for Conflict Prevention and Stabilization,Diplomacy & Trade,,None of the above.,"Through this activity, USAID/CPS/OTI's Central America Regional Initiative (CARI)/El Salvador program will support Cristosal's (a human rights organization) efforts to better understand how extortion impacts internal displacement. A highly-qualified research team made up of forensic technicians and specialists, with the required professional certifications to use the ethics data analytics tool, will coordinate with Cristosal to interview 60 families from Panchimalco and other communities that have been internally displaced to research the dynamics of violence and social norms in the communities that have motivated the population to migrate internally. The data analytics tool will eliminate common biases that many surveys encounter in gathering information, which will mitigate collection and analysis errors. The grantee will analyze the data collected to identify the potential utility of early warning systems for IDPs, and how community based responses could support reducing extortion-related violence. The grantee will offer recommendations on how to operationalize their findings through programming.",Interview transcripts,Retired,Neither,,,,2/1/2022,,,,,,,,,,,,N/A,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, "Chemonics International Inc.'s ""Disinformation Monitoring and Sentiment Analysis""",United States Agency for International Development,USAID,USAID/Bureau for Conflict Prevention and Stabilization,Diplomacy & Trade,,None of the above.,"The Transition to New Sudan program, through a program subcontractor, uses AI technology to monitor social media posts for disinformation, and to monitor citizen sentiment analysis.","Reports, visualizations",Retired,Neither,,,,7/16/1905,,,,,,,,,,,,N/A,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, PEER 8-161: A wood species identification tool to aid in compliance and enforcement of Peruvian timber regulations,United States Agency for International Development,USAID,"USAID/Bureau for Inclusive Growth, Partnerships, and Innovation",Diplomacy & Trade,,None of the above.,"The Xylotron, created by the U.S. Forest Service's Forest Products Laboratory, uses artificial intelligence to rapidly identify wood species with the goal to identify timber prohibited from logging under national law or the Convention on the International Trade of Endangered Species (CITES).",The Xylotron uses AI to identify the timber species in a sample by matching a scanned image with the database of DNA data. The research team added 35 Peruvian species to the Xylotron's database as part of this research project.,Retired,Neither,,,,12/1/2021,,,,,,,,,,,,N/A,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, PEER 9-452: Long-term impacts of land-use/land-cover dynamics on surface water quality in Botswana’s reservoirs using satellite data and artificial intelligence methods: Case study of the Botswana’s Limpopo River Basin (1984-2019),United States Agency for International Development,USAID,"USAID/Bureau for Inclusive Growth, Partnerships, and Innovation",Science & Space,,None of the above.,"The PEER project used data-driven AI for quantitative determination of the relationships between land-use and land-cover (LULC) change, socioeconomic development indicators, and climate change and their impacts on water quality and availability within the Limpopo River Basin.","The researchers developed machine-learning models for mapping and quantifying the spatial-temporal LULC change patterns, modeling water quality and quantity in the dam catchments under climate variability and socio-economic factors.",Retired,Neither,,,,3/1/2024,,,,,,,,,,,,N/A,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, BA Cote d'Ivoire Public Social Media Data,United States Agency for International Development,USAID,USAID/Bureau for Global Health (GH),Diplomacy & Trade,,None of the above.,"For Breakthrough ACTION Côte d’Ivoire, we use a software called Meltwater (www.meltwater.com) to extract and organize public social media data from Facebook, Twitter, web-based news, blogs, and similar sources. Meltwater has web crawlers that extract data in real-time (every 2-6 minutes). They have full firehose access to Twitter and Reddit and access to Facebook and other social sources in line with the API restrictions of those companies. We also have a CrowdTangle (www.crowdtangle.com) account, which is Facebook’s (Meta’s) tool for researchers and for social good. Finally, we have a custom Python application which pulls public posts from the Facebook API (through CrowdTangle) and uses machine learning to optimize “topics” from the text data (i.e. groups of terms that reflect certain subjects of discussion on Facebook). In all these cases, artificial intelligence takes the form of natural language processing and image recognition, for example, sentiment analysis and identifying key terms and topics from unstructured data.","Reports, visualizations",Retired,Neither,,,,3/1/2023,,,,,,,,,,,,N/A,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, USING PREDICTIVE ANALYTICS TO IMPROVE CARE,United States Agency for International Development,USAID,"USAID/Bureau for Inclusive Growth, Partnerships, and Innovation",Health & Medical,,None of the above.,"1) identify high-risk patients for targeted intervention - in this specific use case it's children who are at risk of malnutrition but it is meant to eventually be applicable in other settings beyond health, 2) identify frontline health workers / caregivers in need of additional follow up and support from a community health facilitator (in this project it's a community facilitator but outside of this project it would likely be a supervisor in the future)",,Implementation and Assessment,Both,7/11/1905,7/1/2020,10/1/2024,,Developed with contracting resources.,7200AA20FA00023,No,,,No,,No,No,Yes,,N/A,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.","Age, Gender, Disability Status, Other","Caste, religion, parents' education level",Yes,Yes – source code is publicly available.,https://github.com/dimagi/risk-scores,No,,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,Planned or in-progress.,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.","For this particular use case - nutrition outcome monitoring and prediction, one of the key challenges is stable and continued availability of children and their caregivers to enable continuous tracking of changes in their anthropometric measures. From a technical standpoint, the key risk for using AI in any targeted health intervention is the risk that it negatively impacts the baseline intervention (i.e. redirects resources in a way that negatively impacts the program's impact). This risk is avoided by strictly supplementing implementing partners' existing operations.",,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,,,,"For this study, Dimagi has built a predictive algorithm to generate scores to identify those at highest risk for worsening malnutrition outcomes in order to target an additional intervention. The algorithm is run once at the beginning of the study to identify those to enroll in the top-up intervention, the efficacy of the model is being monitored and evaluated via an RCT.",,,,,, Smart Supervision Platform - Superintendencia Financiera de Colombia (Colombian Financial Regulator),United States Agency for International Development,USAID,USAID/Colombia,Diplomacy & Trade,,None of the above.,"This platform collects data from customer and financial intermediaries about financial products complaints. The system processes all these data and make analysis anticipating trends and other types of components must be improved by the financial institutions and give tools for facilitating the decision – making for supervision activities. The functionality of performing data analysis is based on the information collected using panels control and indicators, it will allow classify the information of the different dimensions, analyzing them statistically, in order to be able to determine trends over time, by entities, regions, and the improvement of complaint or claim attention processes.","Time series predictions, visualizations",Retired,Neither,,,,7/14/1905,,,,,,,,,,,,N/A,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Feed the Future Innovation Lab for Applied Wheat Genomics,United States Agency for International Development,USAID,"USAID/Bureau for Resilience, Environment, and Food Security",Science & Space,,None of the above.,The AI simultaneously utilizes multi-modal deep learning to leverage both genotype and phenotype measurements by fusing genetic variants with multiple sources of data collected by unmanned aerial systems to predict wheat yield in seen and unseen environments.,Testing (the Innovation Lab ended and there is no output to report),Retired,Neither,,,,7/15/1905,,,,,,,,,,,,N/A,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Market segmentation,United States Agency for International Development,USAID,USAID/Bureau for Global Health (GH),Diplomacy & Trade,,None of the above.,"Use of analytical approaches to increase the understanding of the market size and characteristics for private HIV treatment and services. Utilizes existing data to profile people living with HIV/AIDS (PLHIV) accessing HIV services and treatment based on willingness to pay (WTP) to create profiles incorporating demographic dimensions, geographic dimensions, and temporal dimensions to help define the size of the market and for HIV treatment services for PLHIV. Data sources include geospatial datasets, market data on comparable products, and micro-economic data. The use of novel data sources creates outputs in the form of patient profiles or segments at the global, regional, and/or country level to better understand which patients would be willing to pay for HIV treatments or services. Use of the market segmentation activity with development of a country-agnostic approach helps profile and estimate the size of a specific population group of PLHIV based on their willingness to pay for HIV services that can be applied to any PEPFAR-priority country context.","Reports, visualization",Retired,Neither,,,,7/16/1905,Developed with contracting resources.,7200AA19CA0004,No,,,No,,No,No,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,N/A,Yes,No – agency does not have access to source code.,,,,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Climate-based malaria forecast,United States Agency for International Development,USAID,USAID/Bureau for Global Health (GH),Health & Medical,,None of the above.,The climate-based model takes malaria confirmed cases and eco-climatic data to forecast the number of malaria cases/incidences up to 2 months ahead.,Time series predictions ,Initiated,Neither,1/1/2022,,,,,,,,,,,,,,,N/A,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Covid Decision Support,United States Agency for International Development,USAID,USAID/India,Health & Medical,,None of the above.,"USAID-supported TRACE TB led by Wadhwani Institute for Artificial Intelligence supported the Municipal Corporation of Greater Mumbai (MCGM) with eight forecasts commissioned during June 2020 to January 2021. These predictive modelling and analytics forecasts, and recommendations supported the MCGM public health authorities and led to an additional over 2,500 ICU beds and over 50,000 quarantine beds months in advance of the second COVID-19 wave. Based on insights from data analysis the forecast suggested to increase case discovery by ~25% in under-tested wards. Similar support was provided to Jharkhand (six districts) from September 2020 - January 2021 with three forecasts and data analytics. The model received appreciation letters from the State Government of Jharkhand and Maharashtra.",Time series predictions,Retired,Both,,,,7/14/1905,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Artificial Intelligence for observing Hand Hygiene,United States Agency for International Development,USAID,USAID/India,Health & Medical,,None of the above.,"USAID-supported Momentum Country and Global Leadership (MCGL) - project led by Jhpiego, focuses on innovative digital solutions to ensure compliance with WHO-recommended “Nine steps for correct handwashing techniques”. MCGL has developed SafeHands - an Artificial Intelligence tool to monitor and improve facility-level hand hygiene. SafeHands uses AI algorithms to closely watch the movement of palms and fingers in real-time, checks for the handwashing steps prescribed by WHO and measures the time duration for the rubbing action with the soap.",Video annotations,Retired,Neither,,,,7/14/1905,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Cough Against Covid,United States Agency for International Development,USAID,USAID/India,Health & Medical,,None of the above.,"USAID supported TRACE TB led by Wadhwani Institute of Artificial Intelligence (WIAI) developed an AI-based cough sound analysis technology to help identify at-risk Covid-19 patients before administering lab-based Reverse Transcription–Polymerase Chain Reaction (RT-PCR) tests, even if they are asymptomatic. The application uses machine learning algorithms/ deep learning",Sound classification,Retired,Both,,,,7/14/1905,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Automated interpretation of Line Probe Assay strips using computer vision,United States Agency for International Development,USAID,USAID/India,Health & Medical,,None of the above.,"The AI automates the interpretation of Line Probe Assay (LPA) results, enhancing accuracy, efficiency, and consistency in detecting drug-resistant TB. It reduces manual errors, accelerates diagnosis, and integrates with the Ni-kshay platform to support timely treatment decisions for TB patients.","The AI system detects the bands present on the LPA strip and outputs the sensitivity to different drugs, automatically classifying the test results as resistant or susceptible. It generates a final diagnostic report that helps clinicians make informed treatment decisions based on the presence or absence of drug resistance",Acquisition and/or Development,Both,6/1/2020,7/1/2020,,,Developed with contracting resources., 72038620CA00006,No,,,No,,No,No,Yes,,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,N/A,Yes,"Yes – agency has access to source code, but it is not public.",,,,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,No,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.","- Misinterpretation of Results: Incorrect interpretation of AI outputs could lead to delayed or incorrect treatment. This was identified through validation studies and expert reviews. - Integration Disruption: The AI model is hosted in Ni-kshay (Official Health Management Information System for TB in India). Any outage or disruption in Ni-kshay could impact the AI's availability and functionality. This risk was identified during system integration and stress testing.",Yes – by another appropriate agency office that was not directly involved in the system’s development,"Established Process of Machine Learning Operations: Alongside automated testing and drift detection, model re-training and re-deployments are supported by continuous integration pipelines that are managed by machine learning and data engineers on the platform, adapting work done by data science team into repeatable scripts for re-training and re-testing a model once deployed.",No - Some individual decisions or actions require direct human oversight.,,N/A - individuals are not interacting with the AI for this use case,,No,,,,,,, Prediction of risk for Loss to Follow Up (LFU) among TB patients using AI,United States Agency for International Development,USAID,USAID/India,Health & Medical,,None of the above.,"The AI predicts the likelihood of adverse treatment outcomes in TB patients, such as treatment failure or mortality. By identifying high-risk patients early, It facilitates timely interventions, improves patient outcomes, optimizes resource allocation, and enhances the effectiveness of TB treatment programs.","AI model outputs a risk score for each patient that is converted to a HIGH-RISK or LOW-RISK label after comparing it with a district-wise threshold. This helps prioritize patients for timely interventions, improving treatment outcomes and reducing the risk of adverse events",Operation and Maintenance,Both,6/1/2020,7/1/2020,3/1/2023,,Developed with contracting resources., 72038620CA00006,No,,,No,,No,No,Yes,,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ","Age, Gender, Other","Location, Height, Weight",Yes,"Yes – agency has access to source code, but it is not public.",,,,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,No,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.","- Trust: There are two issues related to trust: a. the possibility of trusting model outputs over human judgement could lead to missing of some high risk cases b. Model interpretability poses a challenge, as the algorithms may hinder healthcare professionals' ability to trust or validate predictions. - Accuracy: The solution may produce inaccurate or inequitable predictions due to incomplete or unrepresentative data. This could lead to underperformance on certain sensitive cohorts, further affecting trust in AI-generated outcomes. - These risks were identified and addressed through model evaluation, regular internal data audits and routine monitoring of pilot deployments.",,"Established Process of Machine Learning Operations: Alongside automated testing and drift detection, model re-training and re-deployments are supported by continuous integration pipelines that are managed by machine learning and data engineers on the platform, adapting work done by data science team into repeatable scripts for re-training and re-testing a model once deployed.",No - Some individual decisions or actions require direct human oversight.,,N/A - individuals are not interacting with the AI for this use case,,,,,,,,, Leveraging Artificial Intelligence for TB screening,United States Agency for International Development,USAID,USAID/India,Health & Medical,,None of the above.,"The largest gap in the TB care cascade is the diagnostic pathway, whereby patients are unable to access a TB test and receive proper diagnosis. Seeking initial care from an informal provider and consulting multiple health care providers are consistently associated with diagnostic delays. USAID-funded World Health Partners (WHP) Closing the Gaps in TB Care Cascade (CGC) project is partnering with Deeptek.ai to deploy the Genki product, an automated AI chest X-ray reading tool that can detect 20 abnormalities, enabling healthcare providers to reach a probable diagnosis much earlier. The AI-enabled screening solution aims to overcome and address TB care cascade gaps and achieve fundamental change to support the Government of India's (GoI's) vision of a 'TB-Free India'.",Image classification,Retired,Both,,,,7/15/1905,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, "Social listening to inform youth programming in Colombia (previously ""Communications Strategy"")",United States Agency for International Development,USAID,USAID/Colombia,Diplomacy & Trade,,None of the above.,"Youth conversations are dynamic and are defined by geographic location, socioeconomic conditions, conflicts with the law, environment, level of vulnerability, gender and ethnicity. Youth develop their voices and identity and mobilize through the media, especially social networks. Active listening methods are a fundamental element to understand their interests, role models, and relevant issues in the youth agenda, in order to design campaigns and communication strategies that are relevant, segmented, and engaging. Consequently, the Youth Resilience Activity (Programa Jovenes Resilientes #EfectoColectivo), leverages Artificial Intelligence tools developed by Audience Insights (in partnership with Twitter and Facebook), using ethical data analysis guidelines (anonymization without personal identifying information). The tool collects and analyzes findings on defined variables (e.g., location, age, followers of an @ or #, interests), allowing us to understand socio-demographic trends and general correlations with greater precision. This allows the Program to creatively develop campaigns and strategies that are highly relevant to young people and their contexts, using distribution channels that they most often consult, thereby increasing the messages’ effectiveness, relevance and engagement with youth. Using the IBM Watson platform, the Audience Insights tool provides information on audience affinity, general interest trends, media following, rankings, and personality profiles. In addition, the Insights tool for Facebook provides general data on the number of people in a specific geography or segmentation, juxtapositions (for example, which other accounts they follow) and percentages of types of devices on which they connect.","Reports, visualizations",Retired,Neither,,,,7/15/1905,Developed with contracting resources.,,No,,,No,,No,No,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.","Age, Other","Location, social media use metrics",Yes,No – agency does not have access to source code.,,,,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, AI Pilot- Detection of data exfiltration,United States Agency for International Development,USAID,USAID/Bureau for Management,Mission-Enabling,,None of the above.,"The AI Pilot Project was developed to demonstrate the use of AI at USAID through the detection of security events and anomalies in network data. While existing cybersecurity measures catch some attacks, it is highly likely that many are sneaking by undetected, either disguised as benign traffic or simply unnoticed in the mass of data. The objective of the AI Pilot was to develop an in-house capability to detect previously undetected anomalous actors, trends or events in network data through the use of machine learning algorithms. For the proof of concept, the machine learning models were constructed through reverse engineering. This means the attacks were found and labeled in the data before the models were created, allowing the AI team to evaluate the performance of the machine learning models. The attacks were detected through calculating statistics specific to the type of attack and data protocols used by DHS and identifying outlier values.","ML model that successfully detected known penetration attacks conducted on specific protocols ICMP, HTTP, HTTPS. Discontinued before testing on second data set.",Retired,Neither,,,,8/1/2021,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Global Antimicrobial Resistance Surveillance System,United States Agency for International Development,USAID,USAID/Bureau for Global Health (GH),Science & Space,,None of the above.,"WHO’s Global Antimicrobial Resistance Surveillance System (GLASS) was developed to support global surveillance and research in order to strengthen the evidence base on antimicrobial resistance (AMR) and help inform decision-making and drive national, regional, and global actions. USAID's Infectious Disease Detection and Surveillance (IDDS) project assisted Ministries of Health in Kenya, Cameroon and Liberia to use GLASS. This support included training for surveillance sites to enhance AMR data collection, analysis, interpretation, and reporting to GLASS. This also included regular supportive supervision.","Reports, visualizations",Retired,Neither,,,,5/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, GXAlert,United States Agency for International Development,USAID,USAID/Bureau for Global Health (GH),Health & Medical,,None of the above.,"System One's GXAlert/Aspect software platform integrates with diagnostic devices to bring test results to clinicians, partners and program staff through a private and secure communications network and flexible reporting mechanisms. USAID's IDDS project supports the implementation of the GXAlert software in Zimbabwe, Tanzania and Vietnam.","Reports, test results",Retired,Both,,,,5/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, DataToCare,United States Agency for International Development,USAID,USAID/Bureau for Global Health (GH),Diplomacy & Trade,,None of the above.,"DataToCare is a fully customizable suite of integrated applications (mobile, desktop, web) that connects national laboratory networks by collecting diagnostic and patient data at the facility level, displaying real-time information on a dashboard to facilitate decision-making at the central level, and notifying test results to medical teams and patients. USAID's IDDS project supports implementation of DataToCare in DR Congo and Cambodia.","Reports, visualizations",Retired,Both,,,,5/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, CAD4TB: AI-Powered Tuberculosis Detection Software,United States Agency for International Development,USAID,USAID/Bureau for Global Health (GH),Diplomacy & Trade,,None of the above.,"Computer-Aided Detection for Tuberculosis. USAID's Infectious Disease Detection and Surveillance (IDDS) project is supporting the introduction of Delft's CAD4TB software in DR Congo, Kenya, Nigeria, Philippines, Uganda and Vietnam to improve the reading of chest x-rays for tuberculosis diagnosis.",Image classification,Retired,Both,,,,5/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, DeepGreen Ukraine,United States Agency for International Development,USAID,USAID/Ukraine,Energy & the Environment,,None of the above.,"Project aimed at the development of the Illegal Logging and Deforestation Alert System in Ukraine (𝛃-version) based on the results of the DeepForest Project (created 𝛂-version of the system). It is an IT-based service that will allow its users to detect potential illegal and monitor planned forest logging. The system intends to analyze satellite data on forest cover changes and compare it with open source data published by public institutions (e.g. state land use and ownership maps, forest maps, logging tickets, etc.). The service will reflect near real-time deforestation changes indicating how many forests disappeared from the forest map of Ukraine, when and where it happened, and whether there are appropriate government permits (logging tickets) for felling. ",Annotated maps,Retired,Neither,,,,3/1/2022,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, "National Park information classification using AI, Garamba National Park, DRC",United States Agency for International Development,USAID,USAID/Democratic Republic of the Congo,Science & Space,,None of the above.,"Integrated Python scripting to train the J2/intelligence office workflow on how information from park ranger patrols should be classified, separated, then shared from the confidential data.",Text classification,Retired,Neither,,,,9/30/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, "Wildlife identification using computer vision (Mbaza), DRC",United States Agency for International Development,USAID,USAID/Democratic Republic of the Congo,Science & Space,,None of the above.,"Mbaza was a machine learning program for identifying species in camera traps, which speeds up inventory of biodiversity surveys in a protected area of interest.",Image classification ,Retired,Neither,,,,5/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, AI-enabled presentation design,United States Agency for International Development,USAID,USAID/Democratic Republic of the Congo,Diplomacy & Trade,,Creating visually appealing presentations using AI-driven design suggestions.,Staff time savings and improved quality of presentation materials,Powerpoint presentations,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,N/A,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, "AI tools for learning, evaluation, and research",United States Agency for International Development,USAID,"USAID/Bureau for Democracy, Human Rights, and Governance",Diplomacy & Trade,,Summarizing the key points of a lengthy report using AI.,"More effective research, evaluation, and reporting by a USAID implementing partner",Primarily text; eventually synthesized into reports for USAID activity managers,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,N/A,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, AI-enabled classification for Standardized Program Structure and Definitions,United States Agency for International Development,USAID,USAID/Bureau for the Middle East,Diplomacy & Trade,,None of the above.,"Time-saving automated categorization of information into pre-established definitions. U.S. foreign assistance is categorized using a specific system known as the Standardized Program Structure and Definitions (SPSD). This is important for many reasons. First, the SPSD is comprised of broadly agreed-upon definitions for foreign assistance programs, providing a common language to describe programs. Second, by utilizing a common language, information for various types of programs can be aggregated within a country, regionally or globally allowing for the comparison and analysis of budget and performance data.",Indicators are sorted into the Standardized Program Structure and Definitions,Operation and Maintenance,Neither,1/31/2022,5/20/2022,7/15/2022,,Developed with contracting resources.,GS00F280GA/7200AA19M00017,No,,,No,,No,No,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,N/A,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, MENA Assessment Chatbot,United States Agency for International Development,USAID,USAID/Bureau for the Middle East,Diplomacy & Trade,,Summarizing the key points of a lengthy report using AI.,MENA Assessment Chatbot searches a defined repository of USAID-funded assessments to return information in response to questions,A chatbot responds to questions based on its repository of reports and assessments,Implementation and Assessment,Neither,,,,,,,,,,,,,,,,N/A,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Generating reports for program monitoring and evaluation and literature reviews and syntheses to inform activity design and contribute to learning and knowledge management,United States Agency for International Development,USAID,"USAID/Bureau for Inclusive Growth, Partnerships, and Innovation",Diplomacy & Trade,,Summarizing the key points of a lengthy report using AI.,Reducing staff time costs when generating required reports for USAID and improving the comprehensiveness and efficiency of evaluation and literature reviews by contractors,Written reports -- some for internal purposes only and some for external sharing,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,N/A,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, AI tools for grant application application review,United States Agency for International Development,USAID,USAID/Bureau for Humanitarian Assistance,Diplomacy & Trade,,None of the above.,Streamlining the grantmaking process for humanitarian assistance,"Three ML tools which serve the following purposes: 1) assist with review of emergency applications, 2) assist with review of Resilience and Food Security (RFSA) applications, 3) analyze trends in issues letters, which are a component of the RFSA application review process.",Initiated,Neither,9/29/2024,,,,,,,,,,,,,,,N/A,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Natural Language Processing to detect gender-based violence in Kenya,United States Agency for International Development,USAID,USAID/Kenya and East Africa,Diplomacy & Trade,,None of the above.,"This tool will detect patterns of technology-facilitated gender based violence (TFGBV), provide early identification of potential threats, and facilitate automatic and anonymous reporting. The grantee is planning to employ machine learning algorithms for pattern detection, natural language processing (NLP), computer vision, and sentiment analysis.","Reports, visualizations",Acquisition and/or Development,Neither,9/1/2023,4/1/2024,,,Developed with contracting resources.,7200AA22RFA00002,No,,,,,,,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,N/A,,,,No,,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Identifying harmful social media content in Sinhala and Tamil,United States Agency for International Development,USAID,USAID/Sri Lanka,Diplomacy & Trade,,None of the above.,"The activity will develop AI tools that can identify hate speech online, and to design various counter-measures, ranging from streamlined mechanisms to report such content to crafting counter-narratives that can mitigate the negative effects on targeted communities",Reports on harmful content and content for counter narratives ,Acquisition and/or Development,Neither,7/1/2024,7/1/2024,,,Developed with contracting resources.,,No,,,,,,,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,N/A,,,,No,,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Value4Her Connect Digital Platform,United States Agency for International Development,USAID,"USAID/Bureau for Resilience, Environment, and Food Security",Diplomacy & Trade,,None of the above.,"The intended purpose of the AI is to provide personalized learning experiences, offering tutorials, courses, and resources tailored to the needs and interests of women in agriculture. The Value4Her Connect Digital Platform can connect women entrepreneurs, fostering collaboration and the exchange of ideas and best practices. AI will help women entrepreneurs access markets by providing insights into market trends, demand forecasts, and pricing strategies. The expected benefits of leveraging AI on the platform include empowering women with the skills and knowledge needed to innovate and lead in the agricultural sector. This will improve efficiency and productivity and contribute to the economic growth of the women businesses and eventually to the agricultural sector. AI will also be able to analyze vast amounts of data to provide actionable insights, helping women make informed decisions about their farming practices.","The outputs of the Value4Her Connect Digital Platform include: * Personalized recommendations building on tailored educational content based on the user's interests and knowledge level. * Insights into market demand and price trends to help women decide what to invest in * Alerts and updates on the business performance identifying risks and opportunities. * Networking and collaboration tools for discussion and exchange of ideas among young farmers. * Connecting less experienced women with mentors who can provide guidance and support. * Detailed reports on the profitability of various businesses and ventures. * Calculations of the farm's carbon footprint and recommendations for reducing it.",Initiated,Neither,9/1/2024,,,,Developed with contracting resources.,AID-OAA-A-17-00029 (PIATA),No,,,,,,,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,N/A,,,,No,,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, "SERVIR Agriculture and Food Security: SERVIR is using machine learning techniques to interpret satellite data to improve crop area, crop type, crop condition, and yield estimates at the scale of 10s of meters, aggregated to country, district, province, or country.",United States Agency for International Development,USAID,"USAID/Bureau for Resilience, Environment, and Food Security",Science & Space,n/a,None of the above.,"Improve estimates of cropped areas, crop types, crop condition, and crop yield, to inform food security assessments.",geospatial information that can inform agricultural decision-making,Implementation and Assessment,Neither,7/14/1905,7/15/1905,7/16/1905,,Developed in-house.,n/a,No,,,Yes,"documentation, peer review, and open source methods",No,No,Yes,n/a,Free-and-open satellite data from NASA and ESA,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,n/a,Yes,Yes – source code is publicly available.,https://github.com/SERVIR/servir-aces; https://github.com/SERVIR/SERVIR-Applied-Deep-Learning-Book,No,n/a,6-12 months,No,n/a,Yes,n/a,Yes,n/a,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, "SERVIR Water Security: SERVIR is applying deep learning on in situ and global/regional hydrologic forecasts to fill gaps in historic and recent streamflow and river elevation estimates, and improve streamflow forecasts, to a vertical precision of meters to centimeters, potentially for thousands to millions or river reaches worldwide.",United States Agency for International Development,USAID,"USAID/Bureau for Resilience, Environment, and Food Security",Science & Space,n/a,None of the above.,Fill gaps in river elevation and streamflow estimates; to more accurately forecast streamflow in ungauged rivers.,estimates and forecasts that can inform water security decisions,Acquisition and/or Development,Neither,7/14/1905,7/14/1905,7/15/1905,,Developed in-house.,n/a,No,,,,,,,Yes,n/a,Free-and-open satellite data from NASA and GEOGLOWS streamflow model,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,,,,No,n/a,6-12 months,No,n/a,Yes,n/a,Yes,n/a,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, "SERVIR Ecosystem and Carbon Management: Automation of land cover and change mapping, including forest disturbance due to roads and mining. SERVIR is applying machine learning and deep learning techniques on satellite data to automate the classification and identification of land surface activities at the scale of 10s of meters.",United States Agency for International Development,USAID,"USAID/Bureau for Resilience, Environment, and Food Security",Science & Space,n/a,None of the above.,"Improve detection of land cover changes to inform resource management, carbon and greenhouse gas emissions estimates.",geospatial information that can inform ecosystem and carbon management monitoring systems and decisions,Implementation and Assessment,Neither,7/12/1905,7/14/1905,7/15/1905,,Developed in-house.,n/a,No,,,Yes,"documentation, peer review, and open source methods",No,No,Yes,n/a,Free-and-open satellite data from NASA and ESA,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,n/a,Yes,Yes – source code is publicly available.,https://sites.google.com/uah.edu/2020-tensorflow-technical-exch,No,n/a,6-12 months,No,n/a,Yes,n/a,Yes,n/a,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, "SERVIR Weather & Climate Resilience: Short-term and seasonal weather prediction and hazard mapping. SERVIR is mapping floods and landslides with machine learning classification techniques, on the order of 10m to 10km. SERVIR is exploring the performance of weather foundational models versus numerical weather predictions on the 4-10km scale.",United States Agency for International Development,USAID,"USAID/Bureau for Resilience, Environment, and Food Security",Science & Space,n/a,None of the above.,"Improve detection and mapping of hazards and more accurately predict severe weather, to ultimately reduce impacts of climate- and weather-related events.","estimates and forecasts that can inform short-term weather- and subseasonal climate decisions, and climate-related disaster risk reduction",Acquisition and/or Development,Neither,7/14/1905,7/14/1905,7/15/1905,,Developed in-house.,n/a,No,,,,,,,Yes,n/a,Free-and-open satellite data from NASA and ECMWF weather predictions,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,,,,No,n/a,6-12 months,No,n/a,Yes,n/a,Yes,n/a,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, "SERVIR Air Quality and Health: Processing modeled air quality forecasts and assimilating large volumes of data from satellite instruments. SERVIR is downscaling satellite- and model-derived air quality predictions with neural networks, down to 5km spatial resolution.",United States Agency for International Development,USAID,"USAID/Bureau for Resilience, Environment, and Food Security",Science & Space,n/a,None of the above.,"More accurately track and monitor air quality events, their sources, and their impact areas.",geospatial information that can inform air quality and related health monitoring and decisions,Implementation and Assessment,Neither,7/14/1905,7/15/1905,7/15/1905,,Developed in-house.,n/a,No,,,Yes,"documentation, peer review, and open source methods",No,No,Yes,n/a,Free-and-open satellite data from NASA and ESA,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,n/a,Yes,"Yes – agency has access to source code, but it is not public.",n/a,No,n/a,6-12 months,No,n/a,Yes,n/a,Yes,n/a,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Amazonía Mia - AI for environmental law enforcement in Colombia,United States Agency for International Development,USAID,USAID/Colombia,Law & Justice,,None of the above.,"Strengthen the capabilities of the Attorney General's Office for the investigation and prosecution of environmental crimes, with emphasis on deforestation in the Colombian Amazon, by improving technical, technological and human capabilities, facilitating an effective approach to criminal types and identification and prosecution of major drivers of these behaviors.","Reports, visualizations",Operation and Maintenance,"Rights-Impacting ",7/15/1905,7/15/1905,7/16/1905,,Developed with contracting resources.,72051421D00005,No,,,No,,No,Yes,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,Yes,No – agency does not have access to source code.,,,,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,No,,,,,No - Some individual decisions or actions require direct human oversight.,,,,,,,,,,, Knowledge management and situational awareness for civil society in Colombia,United States Agency for International Development,USAID,USAID/Colombia,Diplomacy & Trade,,None of the above.,"Development of an application that allows social organizations to manage the knowledge they produce in their activities and projects. It seeks, on the one hand, to facilitate access to advanced digital solutions, which enhance the use of information through an intuitive natural language interface, to organizations that require information on security and violence in Colombia that are part of the Suma Social Program. On the other hand, to strengthen organizations that need to monitor security contexts and conditions to support their decision-making processes, with a new digital solution that not only provides, but also facilitates the efficient and effective use of crucial information.","Reports, visualizations, chat interface",Operation and Maintenance,Neither,7/16/1905,7/16/1905,7/16/1905,,Developed with contracting resources.,72051421CA00006,No,,,No,,No,Yes,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,Yes,No – agency does not have access to source code.,,,,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Virtual host for attendees of the Invest for Climate activity,United States Agency for International Development,USAID,USAID/Colombia,Diplomacy & Trade,,None of the above.,Present and interact with the attendees of the Invest for Climate Activity through a virtual host. Objectives include increasing the visibility of the project and facilitating interaction with the attendees.,"Interactive ""host"" avatar",Operation and Maintenance,Neither,7/16/1905,7/16/1905,7/16/1905,,Developed with contracting resources.,72051424D00003,No,,,No,,No,Yes,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,Yes,No – agency does not have access to source code.,,,,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Farmer support chatbot in Colombia,United States Agency for International Development,USAID,USAID/Colombia,Diplomacy & Trade,,None of the above.,Chatbot using AI to interpret agroclimatic information and share with producer families recommend practices based on climate information.,Reports,Operation and Maintenance,Neither,7/16/1905,7/16/1905,7/16/1905,,Developed with contracting resources.,72051423D00002,No,,,No,,No,Yes,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,Yes,No – agency does not have access to source code.,,,,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, "LIDA: Instant Messaging System in the Family Commissioner's Office of Montería - Córdoba, Colombia",United States Agency for International Development,USAID,USAID/Colombia,Government Services (includes Benefits and Service Delivery),,None of the above.,"The LIDA instant messaging system in the Family Commissioner´s Office of Montería, seeks to strengthen communication channels with citizens and strengthen the promotion of rights, based on knowledge of gender-based violence (GBV) referral pathways.",Reports,Operation and Maintenance,Neither,7/16/1905,7/16/1905,7/16/1905,,Developed with contracting resources.,72051422D00001,No,,,No,,No,No,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,Yes,No – agency does not have access to source code.,,,,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, AI assistant for Colombia Partners for Transparency,United States Agency for International Development,USAID,USAID/Colombia,Diplomacy & Trade,,None of the above.,"This tool combines Large Language Models (LLM) with Retrieval-Augmented Generation (RAG) to enhance interaction with the PfT documentation. It allows users to upload documents in word and pdf and ask natural language questions, similar to ChatGPT, providing precise and contextually answers based on the specific content of the documents (https://www.evidencebase.ai/)",Chatbot interface,Operation and Maintenance,Neither,7/16/1905,7/16/1905,7/16/1905,,Developed with contracting resources.,72051420CA00001,No,,,No,,No,No,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,Yes,No – agency does not have access to source code.,,,,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Twitter trend analysis in Colombia,United States Agency for International Development,USAID,USAID/Colombia,Diplomacy & Trade,,None of the above.,"This system uses artificial intelligence through a control dashboard, presenting a counter of retweets in the last hour with gender classification, topic classifier, georeferencing, sentiment trends, and emotion analysis in tweets. This application is designed as a measure to counteract misinformation and promote transparency on social media. In the framework of promoting transparency, access to information, and open government as paths to combat corruption, the product will develop several Machine Learning algorithms for identifying trending posts, recognizing users' demographic, psychographic, and sociographic characteristics, and detecting fake news.","Reports, visualization",Operation and Maintenance,Neither,7/15/1905,7/15/1905,7/15/1905,,Developed with contracting resources.,72051420CA00001,No,,,No,,No,No,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,Yes,No – agency does not have access to source code.,,,,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Project Data Machine: Open governance data in Colombia,United States Agency for International Development,USAID,USAID/Colombia,Diplomacy & Trade,,None of the above.,"This is a technological tool focused on integrating information related to the Congress of the Republic into a homogeneous and easily downloadable format for data extraction or processing. The benefits of this project include (i) Providing data in open and standard formats that are suitable for reuse by the target audience, (ii) Using agreed-upon schemas and vocabularies and providing a defined data structure so that the information can be correctly interpreted when reused, (iii) Offering a consultation point that includes an inventory with descriptive and technical information about the exposed datasets, (iv) Providing accessible data from persistent and user-friendly web addresses, (v) The project serves as a fundamental catalyst for open data initiatives for public and private entities",Text,Operation and Maintenance,Neither,7/15/1905,7/15/1905,7/15/1905,,Developed with contracting resources.,72051420CA00001,No,,,No,,No,No,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,Yes,No – agency does not have access to source code.,,,,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Mibanco: Equitable Finance Activity in Colombia,United States Agency for International Development,USAID,USAID/Colombia,Diplomacy & Trade,,None of the above.,"MICOMUNIDAD Ecosystem: Create a relational ecosystem called Micomunidad, which generates impact for the development and growth of communities, involving different actors that adapt the collaborative work model, promoting financial inclusion, contributing to the reduction of poverty, in a sustainable and easily achievable way, weaving long-term relationships in vulnerable areas. One of the activities is the ChatBot for Financial Education, which is an innovative digital tool that seeks to improve the management of personal and family finances.",Chatbot interface,Operation and Maintenance,Neither,7/15/1905,7/15/1905,7/16/1905,,Developed with contracting resources.,72051422D00003,No,,,No,,No,No,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,Yes,No – agency does not have access to source code.,,,,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Remote sensing for land cover identificiation in Colombia,United States Agency for International Development,USAID,USAID/Colombia,Science & Space,,None of the above.,Training dataset with water body identification capacbility using Landsat and Sentinel Imagery as input,Annotated maps,Operation and Maintenance,Neither,7/16/1905,7/16/1905,7/16/1905,,Developed with contracting resources.,72051423D00003 ,No,,,No,,No,No,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,Yes,No – agency does not have access to source code.,,,,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, AI to support road construction in rural Colombia,United States Agency for International Development,USAID,USAID/Colombia,Diplomacy & Trade,,None of the above.,"An AI-based chatbot that supports the ""Caminos Comunitarios para la Paz Total"" Program seeks to bring to scale the Program's impact across Colombia. By deploying an AI-driven solution, the implementing partner can enhance its outreach and operational efficiency, enabling communities seeking to improve their road networks to access vital information regarding technical and legal requirements quickly through the chatbot solution.",Chatbot interface,Operation and Maintenance,Neither,7/16/1905,7/16/1905,7/16/1905,,Developed with contracting resources.,72051425P00003,No,,,No,,No,No,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,Yes,No – agency does not have access to source code.,,,,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Mobile application for reading rapid cryptococcosis tests,United States Agency for International Development,USAID,USAID/Guatemala,Health & Medical,,None of the above.,"To reduce human error of reading of cryptococcosis tests, the app is used to confirm healthcare worker reading of test results",A positive or negative test result of a cryptoccosis test that a healthcare worker uses to confirm their original diagnosis,Implementation and Assessment,Both,11/1/2023,6/1/2024,12/1/2024,,Developed with contracting resources.,72052023FA00001,No,,,No,,No,No,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,Yes,No – agency does not have access to source code.,,,,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,No,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.",Test-reading error (false positive or false negative),,"Automated and Regularly Scheduled Updates: Some aspects of the plan to monitor the AI system post-deployment are automated including re-training of models after detecting drift; however, data science teams are still significantly involved in the monitoring and re-deployment process.",No - Some individual decisions or actions require direct human oversight.,,In-person,,No,N/A. Photos of test kits with results are the same for all demographics. Patient nor provider demographics are used in the system,Direct user testing,,Yes,,Yes, Chatbot to respond to teachers' questions about how to prevent student dropout,United States Agency for International Development,USAID,USAID/Guatemala,Education & Workforce,,None of the above.,To provide smoother interaction in a chatbot that guides teachers to options for interventions to reduce the likelihood of student dropout,Customized recommendations for teachers based on pre-established protocol of steps to follow to reduce the likelihood of student dropout,Acquisition and/or Development,"Rights-Impacting ",7/1/2024,8/1/2024,,,Developed with contracting resources.,2052022CA00001,No,,,No,,No,No,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,Yes,No – agency does not have access to source code.,,,Dropout Early Warning System Assistant,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,No,"Benchmark evaluation: Testing of the AI model has involved the use of benchmarks (either publicly available or internally created) to estimate performance in real-world settings, but has not been tested in an operational environment. ",Minor risk of chatbot providing incorrect answers to teachers' questions. ,,No monitoring protocols have been established: Necessary infrastructure needed to perform monitoring of AI systems in production is not available and/or a plan to monitor models has not yet been established.,No - Some individual decisions or actions require direct human oversight.,,,,No,No student or teacher infomation interacts with the proposed AI component of the larger app.,Direct user testing,,Yes,,Yes, Predictive analytics model to predict HIV Treatment Interruption Risk,United States Agency for International Development,USAID,USAID/Guatemala,Health & Medical,,None of the above.,Predict HIV Treatment Interruption Risk ,Flagging of patients who may be at risk of dropping out of care,Acquisition and/or Development,Both,4/1/2024,10/1/2024,,,Developed with contracting resources.,7200AA19CA00004,No,,,No,,No,,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",HIV positive individuals,,,,,,,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,,,,,,,,,,No,,,,,,, Anti-Money Laundering and Shareholders Transparency IT System,United States Agency for International Development,USAID,USAID/Moldova,Diplomacy & Trade,,None of the above.,"The implemented Anti-Money Laundering (AML) system currently relies on a rule-based detection framework with advanced networking capabilities. Building on this foundation, additional machine learning (ML) and artificial intelligence (AI) capabilities are being supported to enhance the accuracy of detection logic. This innovative system, the first of its kind in the region, equips employees of the National Bank of Moldova (NBM) with advanced tools to monitor all banking operations comprehensively. The system supports the NBM’s risk-based approach, enabling the identification of suspicious activities and raising of timely alerts. This supports early detection of money laundering and terrorist financing risks, as well as suspicious changes in the ownership structures of banks. Importantly, the NBM's role is supervisory rather than investigative; the system is not designed for law enforcement or prosecutorial purposes. As part of the implementation parameter, a ML/AI proof of concept has been developed. This ML/AI use case specifically focuses on enhancing detection ratios by analyzing alert datasets and generating a segmentation scheme through a K-means clustering algorithm. This segmentation helps identify distinct subgroups (e.g., customers, accounts) with varying levels of money laundering risk, thereby refining the detection process and supporting risk-focused oversight. ","The primary output of the AI algorithm is a segmentation scheme, as described above. This scheme categorizes entities based on their risk levels, enabling improved detection logic and risk assessment within the AML framework.",Operation and Maintenance,Neither,4/1/2021,4/1/2021,8/1/2024,,Developed with contracting resources.,72011719F00001,No,,,No,,No,No,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.","Gender, other","Address, residency, affiliation with different groups, etc.",Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, AI analysis of Moldova's media content and social media landscape,United States Agency for International Development,USAID,USAID/Moldova,Diplomacy & Trade,,None of the above.,"This project utilizes AI to provide analysis of Moldova's media landscape, focusing on narrative framing techniques in both Romanian and Russian-language media and social media. The goal of this application is situational awareness and strategic communications planning, not content removal or blocking.",Monthly Reports,Operation and Maintenance,Neither,10/1/2023,10/1/2023,6/1/2024,,Developed with contracting resources.,AID-117-A-17-00001,No,,,No,,No,No,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,No,No – agency does not have access to source code.,,No,,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, """Bu Mira"" financial inclusion chatbot",United States Agency for International Development,USAID,USAID/Indonesia,Diplomacy & Trade,,None of the above.,"Bu Mira is designed to interface with women entrepreneurs as a business partner on WhatsApp. Through Bu Mira, women entrepreneurs will be able to access educational modules, financial tools, and loan applications.",WhatsApp messages,Operation and Maintenance,Neither,1/25/2022,1/25/2022,1/16/2024,,Developed with contracting resources.,72049722FA00003,No,,,No,,No,No,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",Gender,,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Machine translation for communication with country teams,United States Agency for International Development,USAID,USAID/El Salvador,Diplomacy & Trade,,None of the above.,Machine translation eases communication between the in-country team (which operates entirely in Spanish) and English-speaking colleagues.,Written text,Implementation and Assessment,Neither,6/20/2024,7/15/2024,,,Developed with contracting resources.,"7200AA19D00015, 7200AA24F00002",No,,,No,,No,No,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, E-Audit system for the tax department in Jordan,United States Agency for International Development,USAID,USAID/Jordan,Diplomacy & Trade,,None of the above.,"The system’s focus is on enhancing voluntary compliance, identifying potential high-risk cases for follow-up, and expediting refunds—ultimately supporting both fiscal responsibility and economic growth. By automating routine checks, the e-audit system saves time and resources, which helps the government reinforce public trust in the tax system. This approach also aligns with Jordan's macro fiscal policy objectives, helping generate revenues in a cost-effective way.","A list of scored taxpayers is generated from highest total score to lowest total score. Low scores would indicate low potential for change and thus will be at the bottom rank of risk and most likely will not be audited. Hence, this automated scoring system identifies the tax returns that appear more suspicious than their comparative returns in the same class or sector.",Operation and Maintenance,Neither,2/1/2022,6/1/2022,8/1/2022,,Developed with contracting resources.,72027821C00001,No,,,No,N/A,No,No,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None, ,Yes,No – agency does not have access to source code.,N/A,No,N/A,Less than 6 months,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,Planned or in-progress.,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.","Data clarity and accuracy. PFMA planned for User Acceptance Testing with the counterpart to include all types of taxpayers' profile to ensure the consideration of all possible scenarios.",Planned or in-progress,No monitoring protocols have been established: Necessary infrastructure needed to perform monitoring of AI systems in production is not available and/or a plan to monitor models has not yet been established.,No - Some individual decisions or actions require direct human oversight.,,In-person,,Yes,"The taxpayers profling were idetified as part of the tool design, it refers only to taxpayers size and business type, with no reference to any demographic factors.",Direct user testing,,Yes,,"No – This AI use case is not subject to the opt-out requirement because the AI functionality is solely used for the prevention, detection, and investigation of fraud or cybersecurity incidents, or the conduct of a criminal investigation.", Crop2Cash National Hotline for Agriculture,United States Agency for International Development,USAID,USAID/Nigeria,Diplomacy & Trade,,None of the above.,"This platform is an AI-powered advisory system accessible via phone call. It is available all the time (24/7) in four major Nigerian languages, this virtual extesnion agent provides personalized advice on crop management. pest control, soil health,and market trends. It also integrates real-time weather data and climate predictions, offering adaptive strategies to build rsilience against climate change impacts.",The telephone-based technology allows small holder farmrs access to extension services and information about market trends to impact them mitigate the impact of climate change.,Acquisition and/or Development,Neither,11/2/2024,11/2/2024,,,Developed with contracting resources.,72062020C00001,No,,,No,,No,No,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, FarmerChart,United States Agency for International Development,USAID,USAID/Nigeria,Diplomacy & Trade,,None of the above.,FarmerChart is an AI-powered application that provides agricultural advisory services inclduing weather prediction to smallholder farmers. This platform is accessible via android phones in two major Nigerian languages (Hausa and Yoruba). The system integrates real-time weather data and climate predictions to build farmers resilience against cimate change impacts.,The telephone-based technology allows small holder farmrs access to extension services and information about market trends to impact them mitigate the impact of climate change.,Operation and Maintenance,Neither,4/18/2023,4/18/2023,5/1/2023,,Developed with contracting resources.,72062020C00001,No,,,No,,No,No,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Automated due diligence for corruption risks,United States Agency for International Development,USAID,"USAID/Bureau for Democracy, Human Rights, and Governance",Diplomacy & Trade,,None of the above.,"This platform will process and standardize data from published reports and make it publicly available on a screening platform. Compliance teams at financial institutions and corporations around the world will be able to screen their customers, transactions, and supply chains against the NGO data in the same way that they already screen sanctions and other watchlists.","Reports, visualizations",Initiated,Neither,11/1/2024,,,,Developed with contracting resources.,7200-AA-21D-00024,No,,,No,,No,No,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, AI-powered portal for the Development Experience Clearinghouse,United States Agency for International Development,USAID,USAID/Bureau for Conflict Prevention and Stabilization,Diplomacy & Trade,,None of the above.,Making information in the Development Experience Clearinghouse easier to access and use,Excel tables that feed into a Tableau dashboard with tables and other visualizations.,Operation and Maintenance,Neither,8/1/2023,9/1/2023,1/1/2024,,Developed with contracting resources.,7200AA20D00013,No,,,No,,No,No,Yes,,Documents from the Development Experience Clearinghouse,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,,No,,No,,Other,No relevant resources,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, ARTS - Automated Indirect Cost and Tracking System,United States Agency for International Development,USAID,USAID/Bureau for Management,Mission-Enabling,,None of the above.,"1. Increased Efficiency and Productivity 24/7 Availability: Chatbots can operate continuously, providing support to IPs outside traditional business hours with troubleshooting, FAQs, and routine requests. Rapid Responses: Chatbots can process and respond to IP inquiries immediately providing improved support to IPs around the globe Automation of Repetitive Tasks: AI can handle routine questions, document creation, and workflows, freeing up valuable time for CAS and IP personnel. 2. Enhanced User Experience/Time Savings. Generative AI can tailor trend analysis for indirect rates improving engagement with IPs and reduce the time needed to issue negotiated indirect cost rate agreements. 3. Improved Decision-Making. The CAS Division will use AI to receive data-driven Insights. AI can summarize large datasets of indirect cost rate information to extract insights. Generative AI will synthesize information from the IPs helping negotiators to make informed decisions regarding indirect rate negotiations. AI models for indirect cost rates can predict outcomes and forecast rates based on data provided by USAID IPs. ","Chatbot, Generative AI for trend analysis reports, Indirect Cost Rate Predictions ",Initiated,Neither,10/1/2023,,,,,"47QTCA20D005E / Order # 72MC1023M00003), USAID CIO Enterprise Customer Relationship Management (ECRM) contract",,,,,,,,,,N/A,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Crop pest identification and advisory,United States Agency for International Development,USAID,"USAID/Bureau for Resilience, Environment, and Food Security",Diplomacy & Trade,,None of the above.,To more efficiently and precisely identify crop diseases and provide tailored recommendations on how to control the pest or disease,The AI system generates tailored plant health advisory in the form of text in local languages,Operation and Maintenance,Neither,7/12/1905,7/12/1905,7/15/1905,,Developed with contracting resources.,7200AA21LE00005,No,,,No,,No,No,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, AI-enabled agricultural logistics in Mozambique,United States Agency for International Development,USAID,"USAID/Bureau for Resilience, Environment, and Food Security",Diplomacy & Trade,,None of the above.,"Appload is improving accessibility to logistics services for small agricultural aggregators and small transporters in Nampula and Zambezi provinces and the Beira Corridor via Appload's digital marketplace. Through the partnership, Appload will reach 1,588 shippers and 740 small transporters, facilitating the movement of over $675,000 in cargo through 860 unique trips on the platform. Additionally, in partnership with Hollard Insurance, the company is subsidizing goods in transit insurance for all shippers using the app.","App interface, supply chain predictions",Operation and Maintenance,Neither,10/15/2023,10/14/2025,10/30/2023,,Developed with contracting resources.,7200AA20C00054,No,,,No,,No,No,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Using AI to detect information manipulation,United States Agency for International Development,USAID,"USAID/Bureau for Democracy, Human Rights, and Governance",Diplomacy & Trade,,None of the above.,To detect narratives and trends in social media alterations of images and video in order to identify information manipulation.,"Reports, visualizations",Operation and Maintenance,Neither,12/1/2020,12/1/2020,7/1/2023,,Developed with contracting resources.,7200AA18CA00059,No,,,No,,No,No,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, HelloTask: multichannel job platform for blue-collar workers,United States Agency for International Development,USAID,USAID/Bureau for Asia,Diplomacy & Trade,,None of the above.,"Under the Market Systems & Partnerships mechanism, USAID/Asia is supporting a partnership with HelloTask, a multichannel job platform for blue-collar workers that connects clients to care service providers (e.g., cooking, cleaning, laundry) via a smartphone application. As part of this activity, USAID is providing funding to support the development of an AI-based interactive voice response (IVR) technology that allows users to speak into their phones to interact with the job placement functions of the app.","The telephone-based IVR technology will allow lower-income users to better interface with the online technology platform to accept jobs and provide feedback. Users often do not have smartphones, but instead feature phones that have limited capability to support more sophisticated mobile apps often used by gig workers in other countries. The IVR system currently allows domestic workers to speak directly into their featurephones with the message being transcribed into the online tracking system. The next phase is to allow domestic workers to verbally accept and decline jobs, file feedback, and otherwise interact with the company in a way that is similar to other apps with gig employment.",Acquisition and/or Development,Neither,8/1/2023,3/1/2024,,,Developed with contracting resources.,7200AA20C00054,No,,,,,,,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,,,,No,,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Georgian speech-to-text transcription,United States Agency for International Development,USAID,USAID/Georgia,Diplomacy & Trade,,None of the above.,"The purpose of the tool is to transcribe spoken speech via automatic speech recognition (ASR). The expected benefit, based on use-cases, is to help users transcribe large amounts of audio data into text with enterprise accuracy and in a short amount of time.",Text (spoken audio content),Operation and Maintenance,Neither,9/22/2020,10/1/2021,12/30/2022,,Developed with contracting resources.,72011420CA00002,No,,,No,,No,No,Other,Program collected and labeled the dataset for purpose of training the system to learn to transcribe the speech in Georgian language; repository not currently maintained,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,Yes,No – agency does not have access to source code.,N/A,No,,Less than 6 months,No,,Other,Our partner organization has in-house computing resources for model training and development,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Skhivi: AI tools for Georgian journalists,United States Agency for International Development,USAID,USAID/Georgia,Diplomacy & Trade,,None of the above.,"Skhivi is designed for journalists, news editors, and media researchers and offers a news aggregator, trend analysis, a news distribution timeline across various media outlets, polarization metrics, and a temperature index. The team is now accepting subscription requests for more advanced features, and the tool is already being used by local Georgian media outlets like Tabula and Aprili.",Online dashboard with visualizations and reports,Implementation and Assessment,Neither,8/1/2022,4/1/2024,6/9/2024,,Developed with contracting resources.,72011422CA00004,No,,,No,,No,No,,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,Yes,No – agency does not have access to source code.,,,,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Intelligent forecasting for Family Planning products (Cote d'Ivoire),United States Agency for International Development,USAID,USAID/Bureau for Global Health,Diplomacy & Trade,,None of the above.,Optimizing health supply chains to reduce costs and improve quality of care by avoiding stock-outs.,Supply chain predictions,Implementation and Assessment,Neither,7/12/1905,7/13/1905,5/1/2023,,Developed with contracting resources.,7200AA20CA00009,No,,,No,,No,No,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,Yes,No – agency does not have access to source code.,,,,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Intelligent forecasting for Malaria products (Cote d'Ivoire),United States Agency for International Development,USAID,USAID/Bureau for Global Health,Diplomacy & Trade,,None of the above.,Optimizing health supply chains to reduce costs and improve quality of care by avoiding stock-outs.,Supply chain predictions,Acquisition and/or Development,Neither,7/12/1905,7/14/1905,,,Developed with contracting resources.,7200AA20CA00009,No,,,No,,No,No,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,Yes,No – agency does not have access to source code.,,,,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Intelligent forecasting for Routine Vaccines (Serbia),United States Agency for International Development,USAID,USAID/Bureau for Global Health,Diplomacy & Trade,,None of the above.,Optimizing health supply chains to reduce costs and improve quality of care by avoiding stock-outs.,Supply chain predictions,Implementation and Assessment,Neither,7/15/1905,4/1/2024,8/1/2024,,Developed with contracting resources.,7200AA20CA00009,No,,,No,,No,No,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,Yes,No – agency does not have access to source code.,,,,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Intelligent forecasting for COVID-19 vaccines (Mali),United States Agency for International Development,USAID,USAID/Bureau for Global Health,Diplomacy & Trade,,None of the above.,Optimizing health supply chains to reduce costs and improve quality of care by avoiding stock-outs.,Supply chain predictions,Implementation and Assessment,Neither,9/1/2022,10/1/2022,5/1/2023,,Developed with contracting resources.,7200AA20CA00009,No,,,No,,No,No,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,Yes,No – agency does not have access to source code.,,,,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Intelligent forecasting for TB products (Indonesia),United States Agency for International Development,USAID,USAID/Bureau for Global Health,Diplomacy & Trade,,None of the above.,Optimizing health supply chains to reduce costs and improve quality of care by avoiding stock-outs.,Supply chain predictions,Acquisition and/or Development,Neither,8/1/2022,8/1/2022,,,Developed with contracting resources.,7200AA20CA00009,No,,,No,,No,No,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,Yes,No – agency does not have access to source code.,,,,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Drip irrigation system with AI control,United States Agency for International Development,USAID,USAID/Bureau for the Middle East,Diplomacy & Trade,,None of the above.,"GEAR Center's smart irrigation system uses artificial intelligence to predict local weather, enabling the system to optimize water and energy use. This caters specifically to the unique challenges faced by smallholder farmers in water-stressed regions. By predicting solar exposure and tailoring irrigation needs for individual locations, this technology improves agricultural efficiency, addressing critical issues of soil degradation and water scarcity in regions that can most benefit from these gains.",The AI system outputs a hyper-local weather prediction for the next 24 hours based on historical data and inputs from local weather sensors.,Implementation and Assessment,Neither,7/1/2019,9/1/2019,9/1/2022,,Developed with contracting resources.,7200AA21C00056,No,,,No,,No,No,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Streamlining recruitment with AI-powered online testing,United States Agency for International Development,USAID,USAID/Democratic Republic of the Congo,Diplomacy & Trade,,None of the above.,"TestGorilla is a pre-employment testing platform that enables companies to assess candidates' skills and aptitudes through a variety of tests, including cognitive ability, programming, software proficiency, personality, and culture fit assessments. By focusing on objective skills evaluations, TestGorilla aims to reduce reliance on traditional resumes, thereby minimizing biases and enhancing the efficiency of the hiring process.",Candidate scores and rankings,Initiated,"Rights-Impacting ",10/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Qualitative Analysis of Countering Violent Extremism (CVE) Community Perception Data,United States Agency for International Development,USAID,USAID/Kenya and East Africa,Diplomacy & Trade,,Creating visually appealing presentations using AI-driven design suggestions.,AI-assisted coding of long-form survey responses,Text classification,Retired,Neither,,,,11/1/2024,,,No,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, AI-based translation for key Global Health documents,United States Agency for International Development,USAID,USAID/Bureau for Global Health,Diplomacy & Trade,,None of the above.,"USAID has missions in over 80 countries and programs in more than 100. THis means that there are likely hundreds if not thousands of languages spoken in the countries USAID supports. Translating key health documents and information into all the languages spoken where USAID has programming can be time consuming and expensive. The purpose of this use case is to see if we can make translation cheaper, faster, and eaiser so that more key USAID global health documents are available to partners and populations where we work. Note: no sensitive information is used in this use case.","Two word documents are produced for each translation, one containing just the translated material and one containing a translation report, highlighting any word choice differences or potential inconsistencies.",Acquisition and/or Development,Neither,7/1/2024,7/1/2024,,,Developed in-house.,N/A,No,,,,,,,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,,,,No,,,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Retrieval-augmented generation for government-to-government due diligence,United States Agency for International Development,USAID,"USAID/Bureau for Inclusive Growth, Partnerships, and Innovation",Diplomacy & Trade,,None of the above.,"Government-to-government (G2G) agreements provide a more efficient way for USAID to program funds, but require careful assessment of a partner country's fiscal management and budgetary systems. This application is using AI tools to expedite the initial desk review phase of the G2G Country Context Report by querying a set of authoritative documents with a standardized list of due-diligence questions.","Excel spreadsheet containing columns for questions, answers, and source citations.",Acquisition and/or Development,Neither,2/27/2024,7/11/2024,,,Developed in-house.,N/A,No,,,,,,,Yes,,Completed country context report workbooks from previous G2G assessments were used to benchmark the reliability of the vector search.,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,,,,No,,Less than 6 months,No,,No,,Other,No relevant resources,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, ESDB/IDEA Series Classification,United States Agency for International Development,USAID,USAID/Bureau for Management,Mission-Enabling,,None of the above.,Reduce labor hours involved in classifying new economic and social data indicators to categories in the ESDB/IDEA system by providing automatically suggested categories to data curation staff.,Suggested category assigments for social and economic indicator series.,Initiated,Neither,5/2/2024,,,,,,,,,,,,,,,,,,,,,,No,,,,,,,,,,,,,,,,,,,,,,,,,,,, CDR Document Data Improvement with AI,United States Agency for International Development,USAID,USAID/Bureau for Management,Mission-Enabling,,None of the above.,"Enable improvement of data quality and coverage not acheivable without AI automation given current staffing resources: (1) Filling gaps in summary data, (2) Adding audio/video transcripts (with summaries) (3) Adding descriptions to images. These data improvements will benefit overall user experience, quality of search results, and accessibility.","(1) Text transcripts for video and audio files (2) Text summaries of documents, video files, and audio files (3) Text descriptions of images",Initiated,Neither,5/24/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Evaluation Registry Abstracts Generator,United States Agency for International Development,USAID,USAID/Bureau for Management,Mission-Enabling,,None of the above.,"Automate the generation of USAID evaluation report abstracts for the Evaluation Registry Dashboard based on existing fields in survey data. This automation will streamline the process, reducing recurring labor hours for manually creating the abstract content.",,Initiated,Neither,3/21/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, FADB QA Support Automation,United States Agency for International Development,USAID,USAID/Bureau for Management,Mission-Enabling,,None of the above.,"Reduce data QA and curation level of effort by automating ""first-pass"" of following: (1) Classification of activities by DAC purpose code (2) Identification of countries and regions present in activity descriptions",(1) Suggested DAC purpose codes for a list for activities (2) Suggested countries and regions for activities,Initiated,Neither,6/11/2024,,,,Developed with contracting resources.,72MC1024M00002 ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, TOSSD Data Curation Automation,United States Agency for International Development,USAID,USAID/Bureau for Management,Mission-Enabling,,None of the above.,"Reduce data curation level of effort by automating ""first-pass"" of: (1) Classifcation of activities as ""R&D"" (2) Classifcation of activities as ""Aligned with International Public Good Definitions"" (3) Scoring of activity benefit to TOSSD recipients (4) Scoring of activity compliance with TOSSD guidance (5) Classification of activity to applicable Sustainable Development Goals (SDGs)",(1) Suggested activity category classifications (2) Suggested scoring of activity for assessed benefit and compliance,Initiated,Neither,6/7/2024,,,,Developed with contracting resources.,72MC1024M00002 ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, BHA/DMEAL: Application Review Processes Automation,United States Agency for International Development,USAID,USAID/Bureau for Management,Mission-Enabling,,None of the above.,"Use AI/ML algorithms to perform automated ""first-pass"" reviews and trends analysis of partner-submitted RFSA applications for reduction of burden on BHA staff.","(1) M&E Plan Review Support: flag missing components of the M&E Plan Narrative (e.g. section headers, required detail, etc.) and create summaries of the M&E Plan Narrative sections. (2) RFSA Application Review Support: flag missing components, providing initial scoring of requirements compliance, identifying issues for inclusion in issues letters, summarizing application sections for reviewers, utilizing predictive analytics for objective feasibility and impact assessments, and incorporating data visualization tools to facilitate transparent decision-making. (3) RFSA Solicitation Issues Letters Analysis: use data from Issues Letters and other data sources to identify trends and issues in the solicitation and application review process for past and future Resilience and Food Security Activities (RFSAs). ",Initiated,Neither,9/6/2024,,,,Developed with contracting resources.,72MC1024M00002 ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, BHA/DDAIM: Decision Support via CHOPP,United States Agency for International Development,USAID,USAID/Bureau for Management,Mission-Enabling,,None of the above.,"(1) Implement ""Ask CHOPP"" Feature: Introduce on-demand Gen AI insights with narrative and graphic outputs. (2) Enhance Emergency Application Processing: Improve processes per PLRP requirements to ensure applications meet USAID/BHA guidelines, include all required annexes, and propose activities aligned with humanitarian needs and standards. (3) Improve BHA Situational Awareness: Analyze needs and programming trends over time through enhanced proposal processing. (4) Enrich Contextual Awareness: Blend BHA application needs data with other data sources for a richer understanding of humanitarian contexts. (5) Generate Comprehensive Reports: Produce situation reports and narrative summaries from structured and unstructured data for internal and external reporting, briefings, and historical analyses.","(1) Responses to user inquiries in ""Ask CHOPP"" chat agent (2) Required component inventories and guidance compliance scores. (3) Historical trend analyses for programming trends",Initiated,Neither,9/13/2024,,,,Developed with contracting resources.,72MC1024M00002 ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, GenDev Key Issue Narrative Generation,United States Agency for International Development,USAID,USAID/Bureau for Management,Mission-Enabling,,None of the above.,"A series of AI use cases focused on processing, validating, and summarizing PPR narratives from Facts Info, particularly related to USAID's Gender Development (GenDev) sector. The use cases include: (1) Parsing Narratives by Implementing Mechanism (IM): Extract IM names and corresponding numbers from PPR narratives, and cross-reference them with data reported on operational plan attributions for each IM and Key Issue pulled from the Key Issue Attributions Report on Facts Info. (2) Reviewing and Validating Narratives: Review the PPR narratives against predefined criteria to ensure they meet reporting requirements. This involves flagging whether certain criteria are present, such as correct reporting of IM numbers, description of interventions, and key results. (3) Synthesizing Narrative Review Notes: Synthesize review notes generated by Data Services after the previous use case into concise summaries that are under 500 characters for easy integration into official reports. (4) Keyword-Based Summarization: Combine narratives flagged with similar keywords into harmonized summaries, creating comprehensive narratives on specific themes or issues.","(1) List of extracted IM names and numbers (2) List of binary values for criteria compliance fields (3) Summary of narrative review notes. ",Initiated,Neither,4/17/2024,,,,Developed with contracting resources.,72MC1024M00002 ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, USAID CDM-XDR (Extended Detection and Response),United States Agency for International Development,USAID,USAID/Bureau for Management,Mission-Enabling,,Identifying unusual patterns in system logs from a single incident report using AI.,"Cybersecurity threat detection, threat hunting, and threat response tools to pinpoint vulnerabilities and bolster the security of the Agency's information technology (IT) infrastructure.","Software agents forward security telemetry data to the USAID proxy which directs it to the cloud-based management console, which provides a centralized view and control plane for USAID security analysts.",Initiated,Neither,5/1/2024,,,,Developed with contracting resources.,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Boomitra: Accelerating soil carbon removal on a planetary scale.,United States Agency for International Development,USAID,USAID/Bureau for Humanitarian Assistance,Diplomacy & Trade,,None of the above.,"Boomitra: Accelerating soil carbon removal on a planetary scale. Utilizes satellites and AI to measure soil moisture, nutrients and carbon. Boomitra aids farmers in reducing water and nutrient use by 30 percent while enhancing soil fertility. Funded through WFP Innovation Accelerator","Reports, visualizations",Implementation and Assessment,Neither,4/30/2024,4/30/2024,4/30/2024,,Developed with contracting resources.,720BHA24IO00037,No,,,No,,No,No,Yes,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No resources provisioned by USAID,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, "DARTS: Accurate, accountable cash transfer data and reconciliation",United States Agency for International Development,USAID,USAID/Bureau for Humanitarian Assistance,Diplomacy & Trade,,None of the above.,"DARTS: A web app that uses advanced machine learning to enable WFP country offices to ensure accurate, accountable cash transfer data and reconciliation. DARTS (Data Assurance and Reconciliation Tool Simplified) is a user-friendly web app. It enables WFP to implement controls on large cash transfer data and generate reconciliation reports, which is vital in cash assistance programs. This ensures accountability, evaluating efficiency, building trust and ensuring accurate fund distribution. Funded through WFP Innovation Accelerator",Web app interface,Implementation and Assessment,Neither,4/30/2024,4/30/2024,4/30/2024,,Developed with contracting resources.,720BHA24IO00037,No,,,No,,No,No,Yes,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No resources provisioned by USAID,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, EYouth: Learning and training opportunities for refugees and host communities,United States Agency for International Development,USAID,USAID/Bureau for Humanitarian Assistance,Diplomacy & Trade,,None of the above.,EYouth: Provides learning and training opportunities for refugees and host communities through their AI-based tech learning platform. The venture offers different types of online learning paths through an interactive methodology that helps young people acquire the necessary skills and find the right job. Funded through WFP Innovation Accelerator,Customization of learning content,Implementation and Assessment,Neither,4/30/2024,4/30/2024,4/30/2024,,Developed with contracting resources.,720BHA24IO00037,No,,,No,,No,No,Yes,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No resources provisioned by USAID,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Ignitia: AI-powered weather forceasts for smallholder farmers,United States Agency for International Development,USAID,USAID/Bureau for Humanitarian Assistance,Diplomacy & Trade,,None of the above.,"Ignitia: Uses a reliable and accessible SMS forecast and weather intelligence platform to provide smallholder farmers in the tropics with more efficient weather forecasts. Delivers localized forecasts to farmers through an advanced physics and AI predictive model optimized for the tropics that is twice as accurate as global models. To date, Ignitia has provided services to 2.7 million smallholder farmers, of which approximately 700,000 are recurrent. WFP supports Ingitia with grant funding to further its agribusiness models. This will help them improve agricultural yield. Funded through WFP Innovation Accelerator",Short text messages,Implementation and Assessment,Neither,4/30/2024,4/30/2024,4/30/2024,,Developed with contracting resources.,720BHA24IO00037,No,,,No,,No,No,Yes,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No resources provisioned by USAID,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, SKAI: Large-scale assessment of building damage,United States Agency for International Development,USAID,USAID/Bureau for Humanitarian Assistance,Emergency Management,,None of the above.,"SKAI: An open-sourced tool that uses satellite imagery and AI to automatically assess building damage on a large scale. Enables real-time insights and actionable intelligence for effective decision-making during disaster response. Fuses cutting-edge machine learning algorithms and vast satellite data. Helps organizations make data-driven decisions with unprecedented precision and speed, therefore speeding up humanitarian assistance to affected households after disasters. Funded through WFP Innovation Accelerator","Maps, reports",Implementation and Assessment,Neither,4/30/2024,4/30/2024,4/30/2024,,Developed with contracting resources.,720BHA24IO00037,No,,,No,,No,No,Yes,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No resources provisioned by USAID,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, NEMO: Real-time categorization of hotline calls,United States Agency for International Development,USAID,USAID/Bureau for Humanitarian Assistance,Emergency Management,,None of the above.,"NEMO: Aims to streamline and improve the quality of collected feedback and complaints by using AI to provide real-time categorization recommendations of hotline calls. Employs customized open-source AI algorithms and off-the-shelf AI services. This AI-natural language processing combination improves data collection efficiency through automated speech-to-text for feedback call transcriptions, real-time call categorization assisting hotline operators, and error identification. Funded through WFP Innovation Accelerator",Categorization of calls,Implementation and Assessment,Neither,4/30/2024,4/30/2024,4/30/2024,,Developed with contracting resources.,720BHA24IO00037,No,,,No,,No,No,Yes,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No resources provisioned by USAID,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, "WFP AI Sandbox: Platform to innovate, pilot, and scale AI models and use cases",United States Agency for International Development,USAID,USAID/Bureau for Humanitarian Assistance,Diplomacy & Trade,,None of the above.,"AI Sandbox: Aims to create a collaborative environment that provides WFP colleagues and partners wtih a platform to experiment, innovate, pilot and scale AI models and use cases. Empowers WFP teams to work efficiently and impactfully at scale. Enables the scaling of AI within WFP, piloting impactful use cases with scalability potential. Invests in talent, builds an ethical governance framework, and establishes a responsible-use roadmap. Funded through WFP Innovation Accelerator",Varies by specific use case,Implementation and Assessment,Neither,4/30/2024,4/30/2024,4/30/2024,,Developed with contracting resources.,720BHA24IO00037,No,,,No,,No,No,Yes,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No resources provisioned by USAID,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Enhancing Smart Agriculture in Rural Africa,United States Agency for International Development,USAID,USAID/Bureau for Humanitarian Assistance,Diplomacy & Trade,,None of the above.,"Enhancing Smart Agriculture in Rural Africa: Eliminating post-harvest loss Rooted in real-time data collection, it leverages TinyML and AI to provide actionable insights for farmers. Their system emphasizes early pest and disease detection, ensuring reduced crop losses. Funded through WFP Innovation Accelerator East Africa Regional Hub","Reports, visualizations",Implementation and Assessment,Neither,4/30/2024,4/30/2024,4/30/2024,,Developed with contracting resources.,720BHA24IO00037,No,,,No,,No,No,Yes,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No resources provisioned by USAID,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Emata: Affordable digital finance for farmers,United States Agency for International Development,USAID,USAID/Bureau for Humanitarian Assistance,Diplomacy & Trade,,None of the above.,"Emata: Affordable digital finance for farmers Emata is a Ugandan fintech and licensed microfinance institution, leveraging technology and partnerships with agricultural cooperatives to offer farmers affordable, digital financing. In place of collateral requirements, the company creates AI-powered alternative credit scores based on data points like a farmer’s delivery history. Emata encourages farmers to grow their farms into businesses and invest in a productive future. Funded through WFP Innovation Accelerator East Africa Regional Hub",Credit scores,Implementation and Assessment,Neither,4/30/2024,4/30/2024,4/30/2024,,Developed with contracting resources.,720BHA24IO00037,No,,,No,,No,No,Yes,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No resources provisioned by USAID,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, SoilPro 2.0: Advancing impact monitoring & monetization,United States Agency for International Development,USAID,USAID/Bureau for Humanitarian Assistance,Diplomacy & Trade,,None of the above.,"SoilPRO 2.0, a collaboration with SoilWatch, pioneers in science-based ecosystem and food system restoration and transformation, aims to integrate further innovative environmental monitoring into AIMS by using satellite imagery, peer-reviewed models, machine learning, and in-field testing-based validation. Utilizing the robust data and analysis provided by AIMS for evaluating the impact of our NBS, the project can advance every project activity in the region through a funding and carbon financing toolkit. Funded through WFP Innovation Accelerator East Africa Regional Hub","Reports, visualizations",Implementation and Assessment,Neither,4/30/2024,4/30/2024,4/30/2024,,Developed with contracting resources.,720BHA24IO00037,No,,,No,,No,No,Yes,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No resources provisioned by USAID,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, RAG system for FEWS NET food security reports,United States Agency for International Development,USAID,USAID/Bureau for Humanitarian Assistance,Mission-Enabling,,None of the above.,"The FEWS NET Data, Learning and Communications Hub is developing an AWS SageMaker backed, Retrieval-Augmented Generation (RAG) model utilizing 40 years of historical FEWS NET food security reports. This allows analysts to use natural language to quickly search and retrieve relevant information, with links to the original reports. This model ensures that the wealth of historical FEWS NET food security analysis is accessible. The result is a more informed, data-driven foundation for future project decisions.",Text responses to queries,Acquisition and/or Development,Neither,10/1/2024,10/1/2024,,,Developed with contracting resources.,7200AA24F00008,No,,,No,,No,No,Yes,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No resources provisioned by USAID,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Generating SQL queries from natural language,United States Agency for International Development,USAID,USAID/Bureau for Humanitarian Assistance,Mission-Enabling,,None of the above.,"The FEWS NET Data, Learning and Communications Hub is developing an application that allows select users to generate SQL queries using natural language, based on a modified and secure database schema. By allowing analysts without SQL expertise to efficiently extract data from the FEWS NET Data Warehouse, this tool enhances accessibility and streamlines data retrieval, making critical insights more readily available for decision-making.",SQL queries,Acquisition and/or Development,Neither,10/1/2024,10/1/2024,,,Developed with contracting resources.,7200AA24F00008,No,,,No,,No,No,Yes,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No resources provisioned by USAID,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, API queries from natural language prompts,United States Agency for International Development,USAID,USAID/Bureau for Humanitarian Assistance,Mission-Enabling,,None of the above.,"The FEWS NET Data, Learning and Communications Hub is developing an application that enables a large language model (LLM) to interpret the FEWS NET Data Warehouse OpenAPI specifications, allowing it to generate API queries from user-provided natural language prompts. The LLM can then either extract data from the API or answer questions based on the API responses, facilitating seamless data access and interaction for users without in-depth API knowledge.",Formatted API outputs,Acquisition and/or Development,Neither,10/1/2024,10/1/2024,,,Developed with contracting resources.,7200AA24F00008,No,,,No,,No,No,Yes,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No resources provisioned by USAID,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Interactive exploration of FEWS NET food security reports,United States Agency for International Development,USAID,USAID/Bureau for Humanitarian Assistance,Mission-Enabling,,None of the above.,"The FEWS NET Data, Learning and Communications Hub is implementing a dashboard application developed by the International Center for Tropical Agriculture (CIAT) that uses natural language processing (NLP) to allow uses to interactively explore FEWS NET's 40 years of historic food security reporting to understand the evolution of drivers and trends in food security in the countries that FEWS NET covers.","Reports, visualizations",Acquisition and/or Development,Neither,10/1/2024,10/1/2024,,,Developed with contracting resources.,7200AA24F00008,No,,,No,,No,No,Yes,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No resources provisioned by USAID,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Media synthesis for food security,United States Agency for International Development,USAID,USAID/Bureau for Humanitarian Assistance,Diplomacy & Trade,,None of the above.,"The FEWS NET Data, Learning and Communications Hub is implementing a dashboard application developed by the International Center for Tropical Agriculture (CIAT) that uses natural language processing (NLP) and machine learning to identify to report on relevant topics is both social media (currently Telegram) and formal media (using MedlaCloud) in relevant countries. The media content is evaluated for relvance using a comprehensive taxonomy covering drivers of food insecurity.","Reports, visualizations",Acquisition and/or Development,Neither,10/1/2024,10/1/2024,,,Developed with contracting resources.,7200AA24F00008,No,,,No,,No,No,Yes,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No resources provisioned by USAID,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, "AI translation of water, sanitation, and hygiene learning materials",United States Agency for International Development,USAID,USAID/Bureau for Humanitarian Assistance,Diplomacy & Trade,,None of the above.,BHA WASH-funded partner CAWST uses AI for translation of online WASH learning materials.,Reports ,Initiated,,10/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Labeling and classifying food security activities,United States Agency for International Development,USAID,USAID/Bureau for Humanitarian Assistance,Mission-Enabling,,None of the above.,"Through a buy-in to IPI's Technical Assistance Project for Economic Growth (TAP-EG) award, ALT funded a third Food Assistance and Food Security Assessment (FAFSA-3), which included AI and machine learning models to discover and label each occurence of three project-specific approaches within 3,000 BHA files from resilience food security activities (since 2015) against a taxonomy developed within this project. ",Project classifications,Retired,Neither,5/1/2023,6/25/2023,4/1/2024,6/24/2024,Developed with contracting resources.,47QRAD20DD1072 (TO - 7200AAN00006),No,,,No,,No,No,Yes,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No resources provisioned by USAID,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, SOPHIA: Solutions Platform for Humanitarian analysis,United States Agency for International Development,USAID,USAID/Bureau for Humanitarian Assistance,Diplomacy & Trade,,Searching for information using AI.,"ACAPS, part of the Norwegian Refugee Council (NRC), is a humanitarian information provider that has developed a tool called the Solutions Platform for Humanitarian Analysis (SOPHIA), which uses NLP to filter through hundreds of humanitarian data sources in more than 70 languages, highlighting where to find specific indicators upon request by a user.",Report excerpts,Operation and Maintenance,Neither,1/18/2023,9/26/2023,5/30/2024,,Developed with contracting resources.,720BHA23CA00034,No,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, DEEP: Data Entry and Exploration Platform,United States Agency for International Development,USAID,USAID/Bureau for Humanitarian Assistance,Mission-Enabling,,Summarizing the key points of a lengthy report using AI.,"The Data Entry and Exploration Platform (DEEP), an online platform for storing, sharing, and analyzing humanitarian data, especially unstructured and qualitative data. This includes an AI component of the platform that uses NLP to filter, annotate, and categorize content much faster than a human can. This “tagging” feature is offered to the humanitarian community at large, decreasing the time and effort required to summarize data for decision-makers.","Reports, visualizations",Implementation and Assessment,Neither,6/8/2020,10/1/2021,4/1/2022,,Developed with contracting resources.,720FDA20CA00076,No,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, AI for humanitarian logistics,United States Agency for International Development,USAID,USAID/Bureau for Humanitarian Assistance,Emergency Management,,None of the above.,"Handicap International: Uses AI to create an operational decision support tool based on the automatic analysis of field data. This tool will enable aid providers to develop optimized strategies in the Cabo Delgado and Nampula regions. Models a logistical environment (warehouses, airports, hospitals, schools, etc.) through their interactions, thus forming a complex network. Enables a better understanding of the complex interactions between all the network's constituents. Funded through Humanitarian Grand Challenge","Reports, visualizations",Retired,Neither,6/26/2023,9/24/2024,,12/4/2024,Developed with contracting resources.,720BHA23GR00104,No,,,No,,No,No,Yes,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No resources provisioned by USAID,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, "SMARTplus: digital system for fast, reliable malnutrition data tackling the health-crisis in Somalia",United States Agency for International Development,USAID,USAID/Bureau for Humanitarian Assistance,Health & Medical,,None of the above.,"SMARTplus: digital system for fast, reliable malnutrition data tackling the health-crisis in Somalia, Action Against Hunger ACF (Canada): SMARTplus uses 3D scans to streamline nutrition and mortality data collection. It analyzes and verifies data, aggregating it for visualization on a public dashboard. This prevents delays and reliance on potentially false information. Funded through Humanitarian Grand Challenge","Reports, visualizations",Implementation and Assessment,Neither,6/26/2023,3/14/2024,3/14/2024,,Developed with contracting resources.,720BHA23GR00104,No,,,No,,No,No,Yes,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.","age, gender",,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No resources provisioned by USAID,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, HOPE and HELP: Modeling for complex humanitarian engagement,United States Agency for International Development,USAID,USAID/Bureau for Humanitarian Assistance,Emergency Management,,None of the above.,"HOPE and HELP: Modeling for complex humanitarian engagement, Humanity Data Systems: HOPE and HELP, created by Lensinsight and AllSource Analytics, use advanced technology like Machine Learning and Complex Event Processing to simplify gathering community feedback in humanitarian efforts. They also aid in project and stakeholder management. Funded through Humanitarian Grand Challenge","Reports, visualizations",Implementation and Assessment,Neither,6/26/2023,3/14/2024,3/14/2024,,Developed with contracting resources.,720BHA23GR00104,No,,,No,,No,No,Yes,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No resources provisioned by USAID,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Advanced language technology in marginalized languages for efficient information gathering,United States Agency for International Development,USAID,USAID/Bureau for Humanitarian Assistance,Mission-Enabling,,None of the above.,"Language technology to improve data quality and engagement, Translators without Borders US Inc. aims to create advanced language technology in marginalized languages for efficient information gathering.This pioneering effort could revolutionize understanding of humanitarian needs and priorities in remote regions. Funded through Humanitarian Grand Challenge",App interface,Implementation and Assessment,Neither,6/26/2023,3/14/2024,3/14/2024,,Developed with contracting resources.,720BHA23GR00104,No,,,No,,No,No,Yes,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No resources provisioned by USAID,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Camp forecast: AI-driven supply chain forecasting to reach 7% more refugees without spending more,United States Agency for International Development,USAID,USAID/Bureau for Humanitarian Assistance,Diplomacy & Trade,,None of the above.,"Camp forecast: AI-driven supply chain forecasting to reach 7% more refugees without spending more. The ""Camp Forecast"" (CF) tool employs AI for camp managers to make precise demand forecasts, potentially saving 7% of operational costs. It predicts resident numbers and required supplies in humanitarian camps, pioneering AI-driven forecasting in this context. Funded through Humanitarian Grand Challenge","Reports, visualizations",Implementation and Assessment,Neither,6/26/2023,3/14/2024,3/14/2024,,Developed with contracting resources.,720BHA23GR00104,No,,,No,,No,No,Yes,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No resources provisioned by USAID,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Humanitarian language toolkit: AI-enabled qualitative engagement with conflict populations,United States Agency for International Development,USAID,USAID/Bureau for Humanitarian Assistance,Diplomacy & Trade,,None of the above.,"Humanitarian language toolkit: AI-enabled qualitative engagement with conflict populations, Kobo, Inc.: This project aims to create a toolkit using NLP to record, transcribe, and translate qualitative interviews in humanitarian settings. Field workers can use standard data collection platforms for automatic recording and correction of transcriptions and translations. Funded through Humanitarian Grand Challenge",Text ,Implementation and Assessment,Neither,6/26/2023,3/14/2024,3/14/2024,,Developed with contracting resources.,720BHA23GR00104,No,,,No,,No,No,Yes,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No resources provisioned by USAID,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Chatbot photo diagnosis for malnutrition,United States Agency for International Development,USAID,USAID/Bureau for Humanitarian Assistance,Health & Medical,,None of the above.,"Smart and effective communication in nutrition; ""Chatbot photo diagnosis platform"", Fundación Acción Contra el Hambre (Spain): This project establishes a smart communication channel for effective malnutrition diagnosis and monitoring, a joint effort between the World Food Programme (WFP) and Action Against Hunger (ACF). Funded through Humanitarian Grand Challenge",Chatbot interface,Implementation and Assessment,Neither,6/26/2023,3/14/2024,3/14/2024,,Developed with contracting resources.,720BHA23GR00104,No,,,No,,No,No,Yes,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.","age, gender",,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No resources provisioned by USAID,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, PathVis: A water monitoring device for Vibrio cholerae detection,United States Agency for International Development,USAID,USAID/Bureau for Humanitarian Assistance,Diplomacy & Trade,,None of the above.,"PathVis: A water monitoring device for Vibrio cholerae detection, OmniVis Inc: A groundbreaking smartphone diagnostic tool for rapid V. cholerae detection, providing results in under 30 minutes. Data is instantly sent to a cloud-based server for automated logging. In remote areas, a wifi hotspot enables data upload.This is the first water-based V. cholerae detection device, offering automated, time-stamped, and location-specific data logging. Funded through Humanitarian Grand Challenge",Prediction of V. Cholerae contamination,Implementation and Assessment,Neither,6/26/2023,3/14/2024,3/14/2024,,Developed with contracting resources.,720BHA23GR00104,No,,,No,,No,No,Yes,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No resources provisioned by USAID,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, NeedsBot - Aggregating real-time needs from anywhere via chatbot,United States Agency for International Development,USAID,USAID/Bureau for Humanitarian Assistance,Emergency Management,,None of the above.,"NeedsBot - Aggregating real-time needs from anywhere via chatbot, Needslist: NeedsBot enables vetted frontline responders to text urgent information, supply, and resource needs into the NeedsList platform.This data is then securely aggregated and accessible to multiple stakeholders, facilitating real-time response from individuals, corporations, and governments in the humanitarian aid sector. Funded through Humanitarian Grand Challenge","Reports, visualizations",Implementation and Assessment,Neither,6/26/2023,3/14/2024,3/14/2024,,Developed with contracting resources.,720BHA23GR00104,No,,,No,,No,No,Yes,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No resources provisioned by USAID,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Using technology to combat violence against female refugees,United States Agency for International Development,USAID,USAID/Bureau for Humanitarian Assistance,Diplomacy & Trade,,None of the above.,"Using technology to combat violence against female refugees, ActionAid in London (UK): A low-cost mobile platform for at-risk women and girls to offer essential information on rights, services, safe spaces, and GBV referrals. It enables incident reporting and risk mapping in urban areas. Pilot 1 will launch in collaboration with a Mobile Network Operator and Women's Protection Action Groups. Funded through Humanitarian Grand Challenge","Reports, maps",Implementation and Assessment,Neither,6/26/2023,3/14/2024,3/14/2024,,Developed with contracting resources.,720BHA23GR00104,No,,,No,,No,No,Yes,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No resources provisioned by USAID,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, AutoAnthro: 3D scanning for improved malnutrition assessment in conflict areas,United States Agency for International Development,USAID,USAID/Bureau for Humanitarian Assistance,Health & Medical,,None of the above.,"AutoAnthro: 3D scanning for improved malnutrition assessment in conflict areas, Body Surface Translations, Inc.: Body Surface Translations' AutoAnthro is a cutting-edge 3D scanning system revolutionizing child anthropometry. It swiftly captures precise digital measurements, enhancing outcomes for malnutrition at both individual and population levels. AutoAnthro extracts crucial data, measuring length/height, mid-upper arm circumference (MUAC), and head circumference for children aged 0-5 years. Funded through Humanitarian Grand Challenge",Body measurements of children,Implementation and Assessment,Neither,6/26/2023,3/14/2024,3/14/2024,,Developed with contracting resources.,720BHA23GR00104,No,,,No,,No,No,Yes,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.","age, gender",,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No resources provisioned by USAID,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Machine learning enabled safe water optimization tool for humanitarian response,United States Agency for International Development,USAID,USAID/Bureau for Humanitarian Assistance,Emergency Management,,None of the above.,"Machine learning enabled safe water optimization tool for humanitarian response, York University: This project aims to create a cloud-based tool using AI to optimize safe water practices. By analyzing existing water quality data, it generates customized chlorination instructions for specific field sites worldwide, ensuring safe drinking water based on evidence and site conditions. Funded through Humanitarian Grand Challenge",Text,Implementation and Assessment,Neither,6/26/2023,9/24/2024,9/24/2024,,Developed with contracting resources.,720BHA23GR00104,No,,,No,,No,No,Yes,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No resources provisioned by USAID,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Combatting health misinformation in Yemen,United States Agency for International Development,USAID,USAID/Bureau for Humanitarian Assistance,Diplomacy & Trade,,None of the above.,"Combatting disinformation in conflict zones, The SecDev Foundation: SecDev Foundation collaborates with local partners in Yemen to empower conflict-affected communities in combating harmful health misinformation.They provide guidance through trusted local networks via various channels like emergency alerts, online campaigns, and radio bulletins to ensure accurate information reaches those in need. Funded through Humanitarian Grand Challenge",Text,Implementation and Assessment,Neither,6/26/2023,3/14/2024,3/14/2024,,Developed with contracting resources.,720BHA23GR00104,No,,,No,,No,No,Yes,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No resources provisioned by USAID,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, AI-enabled air raid sirens and war crime documentation,United States Agency for International Development,USAID,USAID/Bureau for Humanitarian Assistance,Emergency Management,,None of the above.,"Hala Systems Inc.: Hala will utilize a network of air-raid sirens administered by civil society, combined with predictive algorithms, training and community engagement efforts to monitor other forms of violence (e.g. artillery, mortars), It will employ blockchain to generate immutable evidence of war crimes. Funded through Humanitarian Grand Challenge",Structured data,Implementation and Assessment,Neither,6/26/2023,3/14/2024,3/14/2024,,Developed with contracting resources.,720BHA23GR00104,No,,,No,,No,No,Yes,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No resources provisioned by USAID,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, AI tools for volunteer mapping,United States Agency for International Development,USAID,USAID/Bureau for Humanitarian Assistance,Diplomacy & Trade,,None of the above.,"Missing maps project, Humanitarian OpenStreetMap Team United States: Missing Maps, a global collaboration, mobilizes volunteers to map uncharted areas through user- friendly tools and open-source apps. Remote volunteers use satellite imagery to create base maps, then local volunteers and agencies refine and verify the details. This hyper-local data supports life-saving crisis response programs. Funded through Humanitarian Grand Challenge",Mapping tools,Implementation and Assessment,Neither,6/26/2023,9/24/2024,9/24/2024,,Developed with contracting resources.,720BHA23GR00104,No,,,No,,No,No,Yes,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No resources provisioned by USAID,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Chatbot to provide accurate COVID-19 information,United States Agency for International Development,USAID,USAID/Bureau for Humanitarian Assistance,Diplomacy & Trade,,None of the above.,"Fighting COVID-19 misinformation with language technology, Translators without Borders US Inc.: The chatbot engages people in their native language, providing accurate pandemic information and informing targeted responses to misinformation. Currently in the Democratic Republic of the Congo,TWB plans to expand to more languages and countries. This innovative solution, accessible via platforms like WhatsApp and Messenger, ensures reliable COVID-19 information reaches vulnerable communities in a language they comprehend. Funded through Humanitarian Grand Challenge",Chatbot interface,Implementation and Assessment,Neither,6/26/2023,3/14/2024,3/14/2024,,Developed with contracting resources.,720BHA23GR00104,No,,,No,,No,No,Yes,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No resources provisioned by USAID,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Imagery authentication from conflict zones,United States Agency for International Development,USAID,USAID/Bureau for Humanitarian Assistance,Diplomacy & Trade,,None of the above.,"Snap truth to power, Sealr: Sealr, a mobile app, combines AI and blockchain to authenticate imagery from conflict zones.This empowers vulnerable populations to report needs securely. As the first verified image-capture app for humanitarian crises, Sealr has the potential to revolutionize a sector often hindered by inefficient feedback systems. Funded through Humanitarian Grand Challenge",Image metadata,Implementation and Assessment,Neither,6/26/2023,3/14/2024,3/14/2024,,Developed with contracting resources.,720BHA23GR00104,No,,,No,,No,No,Yes,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No resources provisioned by USAID,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Tracking COVID-19 misinformation on social media,United States Agency for International Development,USAID,USAID/Bureau for Humanitarian Assistance,Diplomacy & Trade,,None of the above.,"Countering COVID-19 misinformation in areas affected by conflict, Murmurate: Murmurate tracks how misinformation spreads in communities via social media. Using Computer Vision and Natural Language Processing, Murmurate maps COVID-19 conversations online, identifying misinformation that delivers false and damaging public health statements.These insights drive the production and distribution of OVID-19-related counter-messaging. Funded through Humanitarian Grand Challenge","Reports, visualizations",Implementation and Assessment,Neither,6/26/2023,3/14/2024,3/14/2024,,Developed with contracting resources.,720BHA23GR00104,No,,,No,,No,No,Yes,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,No – agency does not have access to source code.,,No,,,No,,No,,Other,No resources provisioned by USAID,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Bambara language processing tool,United States Agency for International Development,USAID,USAID/Mali,Diplomacy & Trade,,None of the above.,"The Bambara language processing tool is being designed to fill a gap that current technology lacks in tracking narratives, usually originating in French here in Mali. Through various platforms, messages are replicated and shared without verification, and at the moment, we're only able to gauge how far French narratives are spreading. This will enable us to track social media narratives in Bambara, a language spoken by 90% of Mali's population.","Reports, visualizations",Acquisition and/or Development,Neither,,,,,,,,,,,,,,Yes,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,,,,No,,,No,,No,,Other,No resources provisioned by USAID,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Repair Spend,Department of Agriculture,USDA,ARS: Agricultural Research Service,Mission-Enabling,,None of the above.,"The intended purpose of this model is to review financial documents and then classify each expense as money spent on ""facility repairs"" or ""not facility repairs"". The expected benefits include reduction of manual hours identifying the types of transactions.","The output of the model is a recommendation of which financial transactions should be identified as ""repair"" expenses.",Operation and Maintenance,Neither,10/1/2019,10/1/2019,6/9/2020,,Developed with both contracting and in-house resources.,Developed by REE-Analytics within EDAPT to refresh Impala tables and visualized on the Tableau Server,No,,,No,,No,No,Yes,,"Approximately 14,000 financial transactions were used to train the model and finetune its parameters. Approximately 3,500 financial transactions were used to test the performance of the final model.","Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,"Enterprise Data Analytics Platform & Toolset (EDAPT, Impala, Tableau)",Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, ARS Project Mapping,Department of Agriculture,USDA,ARS: Agricultural Research Service,Science & Space,,None of the above.,"The intended purpose of this model is to process research plans from various research program portfolios in the Agricultural Research Service (ARS) to find patterns and opportunities between projects. The expected benefits include decreasing the time that humans would spend to manually read, pull out key terms, and group the projects by topic. The model may also find patterns that a human might miss.","The model outputs groups of similar projects and project terms. The output includes metrics (silhouette scores, term rank, importance scores) that show how well the projects and terms in a group match.",Operation and Maintenance,Neither,1/1/2020,1/1/2021,5/1/2022,,Developed with contracting resources.,1232SA21F0067,No,,,No,,No,No,Yes,,"The data is a collection of project plans written by scientists, roughly 600 text documents. The texts are related to publicly available 5-year action plans. ","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Enterprise Data Analytics Platform & Toolset (EDAPT),Less than 6 months,Yes,,No,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, NAL Automated Indexing,Department of Agriculture,USDA,ARS: Agricultural Research Service,Science & Space,,None of the above.,This system automatically assigns word tags to agricultural research articles from a controlled list of terms provided by the National Agricultural Library Thesaurus (NALT). The tags can be used to look up and retrieve articles. Using these tags benefits users by making it easier to find the content they are looking for.,The model outputs terms to use as search tags that are specific to the article that the model analyzed.,Operation and Maintenance,Neither,6/1/2011,1/1/2012,6/1/2012,,Developed with both contracting and in-house resources.,1232SSA1C0014,No,,,Yes,,No,No,Yes,,The data is a collection of text scripts from publishers that have undergone quality assurance and quality control processing.,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,No,,,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Predictive Modeling of Invasive Pest Species,Department of Agriculture,USDA,APHIS: Animal and Plant Health Inspection Service,Mission-Enabling,,None of the above.,"The purpose of the model is to check how likely it is for imported agricultural products from other countries to have pests. Benefits include more reliable discovery and quarantine of invasive pests, preventing pest invasion and making trade safer.",The model outputs are a prediction of whether a product carries an invasive species and what invasive species category the pest belongs to.,Operation and Maintenance,Neither,7/1/2015,7/1/2015,5/1/2018,,Developed in-house.,,No,,,No,,No,No,Yes,,Inspection data was collected from Plant Protection and Quarantine (PPQ) and Customs and Border Protection (CBP). Quality control of the data was conducted by data analysts. There was no data augmention performed. The data is structured and has several million records with more than 50 numerical and categorical variables. This data is not publicly available. ,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Other,Not sure,Other,Not sure,Other,Not sure,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Detection of Pre-symptomatic HLB Infected Citrus,Department of Agriculture,USDA,APHIS: Animal and Plant Health Inspection Service,Science & Space,,None of the above.,The purpose of the model is to detect citrus trees infected with Huanglongbing (HLB) disease using images collected by a camera sensor on a small drone. This system would decrease time and cost associated with manual searching for HLB infected trees.,The model outputs GPS Coordinates of potential Huanglongbing (HLB) infected areas.,Retired,Neither,,,,9/30/2022,,,,,,,,,No,Yes,,,,,,,,,,,,,,,,,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, High Throughput Phenotyping in Citrus Orchards,Department of Agriculture,USDA,APHIS: Animal and Plant Health Inspection Service,Science & Space,,None of the above.,"The main purpose of this system is to analyze drone images to locate, count, and categorize citrus trees in an orchard to monitor orchard health. This use case saves thousands of man-hours searching for signs of plant damage and disease in orchards.",The model output flags images containing plant damage or disease.,Retired,Neither,,,,9/30/2022,,,,,,,,,No,Yes,,,,,,,,,,,,,,,,,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Detection of Aquatic Weeds,Department of Agriculture,USDA,APHIS: Animal and Plant Health Inspection Service,Science & Space,,None of the above.,The purpose of this system is to locate and identify aquatic weed species using images from drones. Expected benefits include decreasing time that would have been spent manually reviewing the images.,The model outputs the aquatic weed species contained in the image.,Retired,Neither,,,,9/30/2022,,,,,,,,,No,Yes,,,,,,,,,,,,,,,,,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Automated Detection & Mapping of Host Plants from Ground Level Imagery,Department of Agriculture,USDA,APHIS: Animal and Plant Health Inspection Service,Science & Space,,None of the above.,This system generates maps of specific tree species from ground-level (streetview) images. Expected benefits are decreased time and cost associated with manual collection of the data.,The model outputs GPS coordinates of flagged locations.,Retired,Neither,,,,9/30/2022,,,,,,,,,No,Yes,,,,,,,,,,,,,,,,,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Democratizing Data,Department of Agriculture,USDA,NASS: National Agricultural Statistics Service,Mission-Enabling,,None of the above.,"This system scans collections of published documents to find how publicly-funded data and evidence are used to serve science and society. This helps the National Agricultural Statistics Service and the Economic Research Service understand who is using their data and why. This improves customer service, helps evaluate programs, and answers important questions for planning and learning.",The model outputs text containing the identified dataset reference information.,Operation and Maintenance,Neither,3/8/2021,6/3/2021,7/30/2021,,Developed with contracting resources.,58-3AEU-3-0003,No,,,Yes,,No,No,Yes,,The model is trained on publicly available peer reviewed research (text data) and known usages of datasets.,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,Yes,Yes – source code is publicly available.,https://github.com/democratizingdata/democratizingdata-ml-algorithms,No,,Less than 6 months,No,,Other,This was a cooperative agreement on all publicly available data. No Federal machines were used to work with the model. ,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Land Change Analysis Tool (LCAT),Department of Agriculture,USDA,FPAC-BC: FPAC Business Center,Mission-Enabling,,None of the above.,"The Land Change Analysis Tool (LCAT) creates high resolution maps to help make land use decisions. For example, it has been used to monitor eastern redcedar for about 40 years in South Dakota and to support wildlife hazard assessments at airports with various organizations. This tool reduced the labor hours needed by the Farm Service Agency (FSA) to review land data accuracy in Georgia by 100 times.",The model outputs land cover maps.,Implementation and Assessment,Neither,10/1/2018,10/1/2018,,,Developed in-house.,,No,,,Yes,,No,No,Yes,,"The dataset is created from the National Agriculture Imagery Program, which contains images of land taken from aircrafts.","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,6-12 months,Yes,,No,,No,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, OCIO/CDO Council Comment Analysis Tool,Department of Agriculture,USDA,OCIO: Office of the Chief Information Officer,Mission-Enabling,,None of the above.,"This prototype helps reviewers identify the main topics and themes of comments, and then group similar comments together. This makes the comment review process more efficient by providing new insights and speeding up comment processing. Benefits include reducing repeated development efforts across the government and saving costs.",The model outputs groups of comments categorized by topic and similarity.,Acquisition and/or Development,Neither,12/1/2020,1/1/2021,,,Developed with contracting resources.,12314418F0020/P00075,No,,,,,,No,Yes,,The training data contains publicly available comment data exported from Regulations.gov. The data largely contains text; some data samples could contain images or data tables. ,"Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",,,,,,No,,Less than 6 months,No,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Retailer Receipt Analysis,Department of Agriculture,USDA,FNS: Food and Nutrition Service,Government Services (includes Benefits and Service Delivery),,Inputting large amounts of data from paper forms into a digital system using AI.,This system uses optical character recognition (OCR) to convert physical inventory documentation into digital text. This makes the review of inventory documents more efficient and consistent.,The model outputs digital text of inventory documentation and distinguishes food items and categories.,Acquisition and/or Development,Neither,10/1/2021,10/1/2021,,,Developed with contracting resources.,Contract number: 47QTCA18D0081,No,,,,,,No,Yes,,The dataset is a collection of Supplemental Nutrition Assistance Program (SNAP) retailer inventory documentation submitted in response to charges of program violations. ,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,,,,No,,More than 12 months,Yes,,No,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Ecosystem Management Decision Support System (EMDS),Department of Agriculture,USDA,FS: Forest Service,Energy & the Environment,,None of the above.,This system provides decision support for environmental analysis and planning by using AI-powered tools in ArcGIS and QGIS. Use of this system empowers stakeholders to make more informed and effective decisions about natural resource management.,"Outputs from this system include the identification of landscapes in need of management/maintenance, along with suggested management actions based on considerations such as cost, efficacy, and policy.",Operation and Maintenance,Neither,1/1/1994,6/1/1995,2/1/1997,,Developed with contracting resources.,Many contracts dating from 1995,No,,,No,,No,No,Yes,,"Primary data sources included observed insect and pathogen-induced mortality, key critical loads for soil and the atmosphere, long term seasonal departures in temperature and precipitation, road densities, uncharacteristic wildfires, historical fire regime departure, wildfire potential, insect and pathogen risk, and vegetation departure from natural range of variability.","Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",,,Yes,No – agency does not have access to source code.,,Yes,Ecosystem Management Decision Support System,6-12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Cross-Laminated Timber (CLT) Knowledge Database,Department of Agriculture,USDA,FS: Forest Service,Energy & the Environment,,None of the above.,"This system enables researchers, practitioners, and the public to find specialized information about timber products. Benefits include faster information sharing and less time spent on manual searches.",System outputs are webpage links from the timber knowledge database.,Operation and Maintenance,Neither,12/1/2017,12/1/2017,6/1/2018,,Developed with contracting resources.,"This was done through a Partnership with the University of Minnesota. Question 35 does not include this option, no contracts were used.",No,,,Yes,,No,No,Yes,,"Public domain agency publications were used for the dataset. This includes station papers, research, and journal papers by US Forest Service and other scientists.","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,No,,Other,Computing done with partner university,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Raster Tools,Department of Agriculture,USDA,FS: Forest Service,Science & Space,,None of the above.,"This system will make machine learning techniques available for geospatial applications. Benefits include standardization of methods, improved work quality, and increased user productivity.","The system API (Application Programming Interface) provides various AI outputs, usually in the form of raster images and data tables.",Implementation and Assessment,Neither,8/1/2021,8/1/2021,,,Developed in-house.,,No,,,No,,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,Yes,Yes – source code is publicly available.,,No,,Less than 6 months,No,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, TreeMap and FuelMap (all versions),Department of Agriculture,USDA,FS: Forest Service; Forest Service Research & Development,Energy & the Environment,,None of the above.,"TreeMap provides a detailed model of the forests in the US. It is used for measuring carbon, planning fuel treatments, starting landscape vegetation models, assessing fire effects, and more. Users include the US Forest Service, private companies, and state governments.",TreeMap produces a detailed map of a plot of forest and a database table listing individual tree records or fuel characteristics for each plot.,Operation and Maintenance,Neither,1/1/2010,10/12/2016,10/1/2018,,Developed in-house.,,No,,,Yes,,No,No,Yes,,"TreeMap is validated using Forest Inventory and Analysis field plot data, consisting of percent forest cover, height, vegetation type, topography (slope, elevation, and aspect), location (latitude and longitude), biophysical variables (photosynthetically active radiation, precipitation, maximum temperature, minimum temperature, relative humidity, and vapour pressure deficit), and disturbance history (time since disturbance and disturbance type) for the landscape around 2016.","Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",,,Yes,Yes – source code is publicly available.,https://github.com/firelab,No,,6-12 months,No,,Yes,,No,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Landscape Change Monitoring System (LCMS),Department of Agriculture,USDA,FS: Forest Service,Science & Space,,None of the above.,This project monitors large areas for changes in land cover and land use over time. The benefits include creating a consistent method for tracking changes in the landscape.,"The model outputs predictions of vegetation gain, vegetation loss, land cover, and land uses.",Operation and Maintenance,Neither,1/15/2016,1/1/2017,3/1/2021,,Developed in-house.,,No,,,Yes,Providing International Organization for Standardization metadata and independent map validation results.,No,No,Yes,,"Training and evaluation data used includes 15,000 sample plots spanning from 1985 - 2019. Features of the dataset are land cover, land use, and change process categories. Data is available upon request.","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Google Cloud Platform contract,6-12 months,Yes,,Yes,,Yes,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Forest Health Detection Monitoring,Department of Agriculture,USDA,FS: Forest Service,Energy & the Environment,,None of the above.,This project monitors forest health by detecting tree damage through changes in light patterns collected by satellites. This detection method helps the Forest Health Protection program monitor areas that can't be checked on the ground or with aerial surveys.,"The model outputs the stage of forest health based on the image, along with a map (polygons) of the area for monitoring.",Implementation and Assessment,Neither,6/29/2021,11/18/2021,,,Developed with both contracting and in-house resources.,1284JC18D005/1284JC20F0036,No,,,No,,No,No,Yes,,"Training data was collected from the National Agriculture Imagery Program (NAIP) and WorldView imagery. Similar high resolution imagery has been used in evaluation and validation. There are approximately 1000 labeled images in the training dataset, each containing spectral data. The data are not publicly available.","Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Cropland Data Layer,Department of Agriculture,USDA,NASS: National Agricultural Statistics Service,Other,Agricultural Statistics,None of the above.,This project produces supplemental estimates of crop acreage and releases geospatial data products to the user community.,The system outputs are an acreage estimate and agrigulture-specific land cover product.,Operation and Maintenance,Neither,1/1/2008,1/1/2008,1/1/2008,,Developed in-house.,,No,,,Yes,,No,No,Yes,,"USDA/Farm Service Agency Common Land Unit data, consisting of Geographic Information System (GIS) shapefiles and associated attribute data, was used for training and testing. ","Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",,,No,No – agency does not have access to source code.,,Yes,See5,Less than 6 months,No,,No,,Yes,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, List Frame Deadwood Identification,Department of Agriculture,USDA,NASS: National Agricultural Statistics Service,Other,Agricultural Statistics,None of the above.,"This model helps identify farms that may be out of business on the National Agricultural Statistics Service list. Parts of the model were used to create clear rules to identify these farms. The resulting list is more accurate and allows for smaller sample sizes, reducing the burden on respondents.",The output of the model was a probability score that a farm is out of business. ,Operation and Maintenance,Neither,2/4/2014,2/3/2016,1/24/2018,,Developed in-house.,,No,,,No,,Yes,No,Yes,,"National Agricultural Statistics Service List frame and survey data, consisting of features put together to form profiles of farming operations, was used to train and validate the model.","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",Age,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,"Research, Education, and Economics (REE) - National Agricultural Statistics Service (NASS) - Citrix Environment",Less than 6 months,No,,Yes,,No,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Climate Change Classification NLP,Department of Agriculture,USDA,NIFA: National Institute of Food and Agriculture,Mission-Enabling,,None of the above.,The Climate Change Classification Natural Language Processing (NLP) model identifies likely climate-related projects within National Institute of Food and Agriculture's (NIFA) large and diverse funding portfolio. Expected benefits include reduced labor hours for reporting and increased repeatability and accuracy of reporting.,"Model output is a list of climate change projects classified as ""climate change related"" or ""not climate change related"" for National Institute of Food and Agriculture (NIFA) internal project review/adjudication and reporting.",Acquisition and/or Development,Neither,7/1/2021,7/1/2024,,,Developed with both contracting and in-house resources.,HHSN316201200002W,No,,,,,,No,Yes,,"This model is trained on publicly available project information provided by the project directors via National Institute of Food and Agriculture's (NIFA) REEport reporting portal. This includes text fields containing the project's title, summary, objectives, and keywords. The training data is catagorized by the project team. ","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,,,,No,,More than 12 months,Yes,,Yes,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Video Surveillance System,Department of Agriculture,USDA,"OSSP: Office of Safety, Security, and Protection",Mission-Enabling,,None of the above.,The purpose of this system is to conduct facial recognition video surveillance to provide enhanced security. Benefits include reduced labor hours for technicians and augmented surveillance capability. ,"The system outputs a positive match to the security control center, indicating identification of the selected individual. An alarm notification is sent to alert security personnel. ",Operation and Maintenance,"Rights-Impacting ",2/14/2020,2/14/2020,2/14/2021,,Developed with both contracting and in-house resources.,12314420A0001,No,,,No,,No,No,Yes,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.","Sex/Gender, Race/Ethnicity",,No,No – agency does not have access to source code.,,No,,Less than 6 months,No,,No,,No,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,No,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.",The key risk of this system is the misidentification of an individual.,Planned or in-progress,No monitoring protocols have been established: Necessary infrastructure needed to perform monitoring of AI systems in production is not available and/or a plan to monitor models has not yet been established.,No - Some individual decisions or actions require direct human oversight.,,None of the above,,Yes,,None of the above,,Yes,,"No – This AI use case is not subject to the opt-out requirement because the AI functionality is solely used for the prevention, detection, and investigation of fraud or cybersecurity incidents, or the conduct of a criminal investigation.", Aquisition Approval Request Compliance Tool,Department of Agriculture,USDA,OCIO: Office of the Chief Information Officer,Mission-Enabling,,None of the above.,This project was developed to help identify likely Information Technology (IT) purchases that do not have an associated Acquisition Approval Request. The benefits are reducing unauthorized IT purchases and increasing compliance with IT procurement procedures and approvals.,The output is a score indicating how likely it is that the purchase is an Information Technology (IT) purchase.,Acquisition and/or Development,Neither,10/31/2019,4/6/2020,,,Developed with both contracting and in-house resources.,12314419D0006,No,,,,,,No,Yes,,The model was developed using text entries entered into USDA's Integrated Acquisition System (IAS).,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,,,,No,,Less than 6 months,No,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Operational Water Supply Forecasting for Western US Rivers,Department of Agriculture,USDA,NRCS: Natural Resources Conservation Service,Mission-Enabling,,None of the above.,"The National Water and Climate Center has a multi-model machine-learning metasystem (M4) for generating water supply forecasts. This model uses AI and other data-science technologies to reduce forecast errors, helping stakeholders make better decisions about water supply availability.",The model outputs water supply forecasts.,Operation and Maintenance,Neither,12/1/2019,12/1/2019,1/1/2024,,Developed in-house.,,Yes,Natural Resources Conservation Service - USDA,Applying for the Environmental Quality Incentives Program (EQIP),Yes,,No,No,Yes,,The data contains snow and precipitation metrics from the Natural Resources Conservation Service (NRCS) Snow Survey and Water Supply Forecast program Snow Telemetry (SNOTEL) monitoring network. ,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,No,,No,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Standardization of Cut Flower Business Names for Message Set Data,Department of Agriculture,USDA,APHIS: Animal and Plant Health Inspection Service,Mission-Enabling,,None of the above.,"Natural Language Processing (NLP) is used to match names of producers to varietal information for cut flowers, which will help convert from manual to automated inspection systems. The main benefit of automation is that it can manage thousands of entities, which would be impossible to handle manually.",The model outputs before and after lists of producer names and cut flower varieties.,Retired,Neither,,,,1/1/2024,,,,,,,,,No,Yes,,,,,,,,,,,,,,,,,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Intelligent Ticket Routing,Department of Agriculture,USDA,OCIO: Office of the Chief Information Officer,Mission-Enabling,,None of the above.,"Help desk tickets are often sent to the wrong group and must be manually re-routed to the correct group, which can take time, resources, and may delay issue resolution. The Intelligent Ticket Routing system helps to send the ticket to the correct group, increasing customer satisfaction by reducing the number of times a customer is transferred or placed on hold, and decreasing the average handle time (AHT). In our specific use case, we reduce the time taken to route a ticket to the appropriate group, shortening the time required to resolve an issue.",The system outputs a prediction of the appropriate group for ticket management.,Operation and Maintenance,Neither,1/1/2022,1/1/2022,1/1/2022,,Developed with both contracting and in-house resources.,ITSS Contract,No,,,No,,No,No,Yes,,"The data is a collection of scripts from Remedy reporting database, a structured database containing freeform text fields. More than 100,000 samples and 526 features comprise the data, including numerical, categorical, text, and date types. Data is not public.","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Midrange + Moving to Sagemaker (ATO in progress),Less than 6 months,Other,DISC / Hybrid Hosting Infrastructure,Other,"DISC / ERMS Portal - SD105 Request, must be DISC employee, or customer.",Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Predictive Maintenance Impacts,Department of Agriculture,USDA,OCIO: Office of the Chief Information Officer,Mission-Enabling,,None of the above.,"A natural language processing (NLP) model classifies whether infrastructure maintenance changes will or will not cause an incident at the Digital Infrastructure Services Center (DISC). Using this system, the business can improve the review process or address specific needs within groups. This will lead to process improvements, increased productivity, higher performance and job satisfaction, higher client satisfaction, and better achievement of key performance indicators (KPIs).",The model outputs a score between 0 to 1. Closer to 1 indicates higher likelihood of an incident created by the proposed change.,Operation and Maintenance,Neither,3/1/2020,3/1/2020,3/1/2020,,Developed with both contracting and in-house resources.,ITSS Contracts,No,,,No,,No,No,Yes,,"The dataset is composed of Remedy data, containing inventory data of all of assets at DISC: hardware, software, incident tickets, and change tickets. Data contains numerical, categorical, text, and date data types.","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Midrange (current) + Sagemaker (ATO in process now - future),Less than 6 months,Other,"DISC / ERMS Portal - SD105 Request, must be DISC employee, or customer.",Other,"DISC / ERMS Portal - SD105 Request, must be DISC employee, or customer.",Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Artificial Intelligence SPAM Mitigation Project,Department of Agriculture,USDA,OASCR: Office of the Assistant Secretary for Civil Rights,Government Services (includes Benefits and Service Delivery),,None of the above.,"An AI/ML model automatically classifies and removes spam and marketing emails from civil rights complaints email channels. Benefits include reducing the time spent manually managing email channels, decreasing the memory burden on email systems, and lowering the risk from malicious emails. ","The model outputs a classification of received emails, flagging spam, marketing, and phishing emails.",Retired,Neither,,,,9/30/2023,,,,,,,,,No,Yes,,,,,,,,,,,,,,,,,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Approximate String Matching (aka fuzzy matching) to Standardize Data,Department of Agriculture,USDA,APHIS: Animal and Plant Health Inspection Service,Mission-Enabling,,None of the above.,"A model is used to replace typos in Plant Protection and Quarantine (PPQ) program data using a list of standardized producer and commodity names. This results in clean, standardized data through an automated workflow. Benefits include reducing labor hours compared to manual data cleaning, makes near-real-time reporting possible, and accurate data enables program managers to conduct efficient policy enforcement and program monitoring.",The model outputs corrected text data.,Operation and Maintenance,Neither,2/1/2023,2/23/2023,5/16/2023,,Developed in-house.,,No,,,No,,No,No,Yes,,"The data used includes data tables from the Agriculture Risk Management (ARM) data system, Agricultural Commodity Import Requirements (ACIR), and the Participating Government Agencies (PGA) Message Set. The cleaned data is displayed in a dashboard for program monitoring. Quality control data are exported from each matching routine and used to validate results as needed.","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Automated PDF Document Processing and Information Extraction,Department of Agriculture,USDA,APHIS: Animal and Plant Health Inspection Service,Mission-Enabling,,None of the above.,This use case takes program and workforce related information stored in thousands of PDFs and converts the information into data tables that can be used for analytics and dashboards. This makes information that is difficult to find available in real-time to support decision making and saves large amounts of time compared to previous methods used.,The model outputs structured database tables.,Operation and Maintenance,Neither,10/15/2022,10/24/2022,1/15/2023,,Developed in-house.,,No,,,No,,No,No,Yes,,"The data used is from Agriculture Vessel Inspection forms concerning flighted spongy moth complex, sent from Customs and Border Protection to Plant Protection and Quarantine.","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,No,No – agency does not have access to source code.,,Yes,Microsoft Power Automate,Less than 6 months,No,,No,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Census Propensity Scores via ML,Department of Agriculture,USDA,NASS: National Agricultural Statistics Service,Other,Agricultural Statistics,None of the above.,This model predicts how likely individuals or operations are to complete the Census of Agriculture. The predictions can help data collectors decide where they need to focus their efforts in order to get more complete census responses.,The model outputs a probability score (all values from and including 0 to 1).,Operation and Maintenance,Neither,10/1/2022,10/1/2022,11/1/2022,,Developed in-house.,,No,,,No,,No,No,Yes,,"The data is collected from the 2017 Census of Agriculture, which includes number of farms by size and type, inventory and values for crops and livestock, producer characteristics, and much more.","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",Other,Zipcode,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,No,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Ecological Site Descriptions (Machine Learning),Department of Agriculture,USDA,NRCS: Natural Resources Conservation Service,Mission-Enabling,,None of the above.,"This AI/ML work conducts analysis of over 20 million records of soils data and 20,000 text documents of ecological information in order to provide complete soil based ecological information for the country. Benefits include reduction in labor hours manually analyzing documents, and enabling stakeholders to examine records in ways previously not thought of to make more informed decisions.",The AI model outputs ecological soil classifications and mappings.,Retired,Neither,,,,9/30/2023,,,,,,,,,No,Yes,,,,,,,,,,,,,,,,,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Conservation Effects Assessment Project,Department of Agriculture,USDA,NRCS: Natural Resources Conservation Service,Mission-Enabling,,None of the above.,"The purpose of the use case is to predict the conservation effects of cropland practices in real time, with no technical skill required. Such models would allow field conservation planners to have real-time conservation effects on sediment and nutrients.",The model outputs predictions of sediment and nutrient change values based on conservation methods.,Acquisition and/or Development,Neither,11/1/2021,11/1/2021,,,Developed in-house.,,No,,,,,,No,Yes,,"The data is collected from the Conservation Effects Assessment Project Survey cropland assessment, which measures trends in cropland conservation practices and their outcomes over time.","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,,,,Yes,R Studio,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Digital Imagery (no-change) for NRI Program,Department of Agriculture,USDA,NRCS: Natural Resources Conservation Service,Mission-Enabling,,None of the above.,"AI algorithms are used to look at landscape images and detect if the landscape has changed from year to year. Currently, about 72000 aerial images are interpreted by dozens of technicians each year to collect data for the National Resources Inventory (NRI) program. This case would decrease the number of labor hours required of technicians to manually interpret images.",The model outputs a classification of “no-changes” in the images if the landscape in the images remain stable from year to year.,Initiated,Neither,10/1/2022,,,,,,,,,,,,No,Yes,,,,,,,,,,,,,,,,,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Nutrition Education & Local Access Dashboard,Department of Agriculture,USDA,FNS: Food and Nutrition Service,Government Services (includes Benefits and Service Delivery),,None of the above.,"The goal of this dashboard is to provide county-level information on nutrition education and local food access, alongside other metrics related to hunger and nutritional health. This interactive dashboard can provide specific details based on the properties of farm to school intensity and size, program activity intensity, ethnicity and race, fresh food access, school size, and program participation. These properties allow users to find similar states based on any of these characteristics, opening up opportunities for partnerships with states they may not have considered. Benefits include increasing stakeholder awareness and empowering more informed decision-making and collaboration.",The model outputs groups of similar counties/states based on the different combinations of properties available for states. ,Operation and Maintenance,Neither,11/9/2022,1/20/2023,6/23/2023,,Developed with both contracting and in-house resources.,,No,,,Yes,,No,No,Yes,,"The data is composed of numerical and categorical data describing farm to school intensity and size, program activity intensity, ethnicity and race, fresh food access, school size, and program participation.","Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",Race/Ethnicity,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Survey Text Remarks Value Scoring,Department of Agriculture,USDA,NASS: National Agricultural Statistics Service,Mission-Enabling,,None of the above.,The purpose of this use case is to analyze a large amout of text in survey responses and score all comments with a priority value. The highly scored blocks of text then get prioritized for review by a human and are responded to more quickly than if they were to be retained in a queue. ,"The model outputs a value score on each snippit of text, highly scored snippits of text are placed at the front of the queue before lower scored blocks to capture text of value more quickly.",Operation and Maintenance,Neither,9/24/2021,12/27/2021,4/20/2022,,Developed in-house.,,No,,,No,,Yes,No,Yes,,The training data is created from historically collected text and survey remarks. ,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,ELMO,Less than 6 months,No,,Other,This is not needed,,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Survey Outlier Detection Model,Department of Agriculture,USDA,NASS: National Agricultural Statistics Service,Other,Agricultural Statistics,None of the above.,The purpose of this use case is to identify abnormal values to edit in surveys. This reduces manual labor and improves data quality.,The model outputs a recommendation of which values in a dataset should be changed.,Operation and Maintenance,Neither,5/1/2022,5/1/2022,5/27/2022,,Developed in-house.,,No,,,No,,No,No,Yes,,"The data is collected from National Agricultural Statistics Service Survey Data, which is structured data containing categorical and numerical values.","Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",,,Yes,Yes – source code is publicly available.,https://jds-online.org/journal/JDS/article/1383/file/13236,No,,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Multilingual Translation of Recalls and Public Health Alerts,Department of Agriculture,USDA,FSIS: Food Safety and Inspection Service,Government Services (includes Benefits and Service Delivery),,None of the above.,"The purpose of this system is to expand the multilingual outreach of food safety information like recalls and public health alerts. Benefits include cost savings on vendor translation services, faster messaging circulation, and increased number of available languages to the general public.",The model outputs multilingual translations created from the original english text.,Acquisition and/or Development,Neither,6/15/2023,6/15/2023,,,Developed with both contracting and in-house resources.,Leveraging Azure's COTS product service,No,,,,,,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,,,,No,,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Genomic Analyses of Pathogen Subtypes,Department of Agriculture,USDA,FSIS: Food Safety and Inspection Service,Science & Space,,None of the above.,"The purpose of this use case is to use machine learning (ML) methods to group foodborne germs based on patterns in their genes, then connect this information with available health data to evaluate foodborne germ risk to public health. Expected benefits include improving our understanding of important foodborne germ genes, assessing key genes and new trends, and identifying and ranking germs that are important for public health.","The model outputs predictions of high risk foodborne germ subtypes, key genetic markers by importance, and emerging trends.",Acquisition and/or Development,Neither,8/1/2022,2/1/2023,,,Developed in-house.,,No,,,,,,No,Yes,,"The data is composed of agency-owned data, including whole genome sequencing (WGS) data from Food Safety and Inspection Service (FSIS). Sampling programs are publicly available and hosted on the National Center for Biotechnology Information (NCBI) database.","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,,,,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Foodborne Illness Source Attribution,Department of Agriculture,USDA,FSIS: Food Safety and Inspection Service,Science & Space,,None of the above.,"The Interagency Food Safety Analytics Collaboration (IFSAC) - a partnership between the Centers for Disease Control and Prevention (CDC), the U.S. Food and Drug Administration (FDA), and the Food Safety and Inspection Service (FSIS) - has used computer-based methods to predict the likely sources of foodborne illnesses in humans caused by various germs (e.g., Salmonella, Campylobacter). Expected benefits include improving our understanding of where these germs come from and how they spread, which can help in creating measures and policies to prevent or reduce illnesses and the overall impact of these diseases.","The model outputs predictions of likely sources of foodborne human illness cases, along with a confidence score of how probable it is that the illness came from the predicted source.",Acquisition and/or Development,Neither,8/2/2021,2/1/2023,,,Developed in-house.,,No,,,,,,No,Yes,,"The data is composed of agency-owned data, including whole genome sequencing (WGS) data from Food Safety and Inspection Service (FSIS). Sampling programs are publicly available and hosted on the National Center for Biotechnology Information (NCBI) database.","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,,,,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Public Comments Analysis,Department of Agriculture,USDA,APHIS: Animal and Plant Health Inspection Service; AMS: Agricultural Marketing Service,Mission-Enabling,,None of the above.,The purpose of the model is to automate the analysis of comments from regulations.gov to help personnel in their review and response tasks. Benefits include a reduction in the number of labor hours needed for review and response.,The model outputs text analysis and categorization of the public comments. ,Acquisition and/or Development,Neither,11/1/2023,11/1/2023,,,Developed in-house.,,No,,,,,,No,Yes,,,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,,,,Yes,Marketing and Regulatory Programs Azure GSS; Marketplace for Data & Analytics (MDA),6-12 months,No,,Other,In development,No,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Rangeland Analysis Platform,Department of Agriculture,USDA,ARS: Agricultural Research Service,Energy & the Environment,,None of the above.,"The Rangeland Analysis Platform (RAP) allows users to track changes in plant growth and coverage over time. By monitoring the condition of agricultural ecosystems and the impact of conservation efforts, it can guide conservation practices for wildlife habitats, carbon assessments, and tax assessments.",The system outputs estimated fractional plant cover and net primary productivity estimates.,Operation and Maintenance,Neither,4/1/2022,3/1/2023,1/1/2017,,Developed in-house.,,No,,,Yes,,No,No,Yes,,"The data is collected from over 75,000 monitoring locations collected by Natural Resources Conservation Service (NRCS), Bureau of Land Management, and other agencies. These data were combined and stored in the Landscape Data Commons (www.landscapedatacommons.org).","Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,"Google Earth Engine, Google Cloud",Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Predictive Cropland Data Layer,Department of Agriculture,USDA,NASS: National Agricultural Statistics Service,Other,Agricultural Statistics,None of the above.,The purpose of this system is to predict crop rotations. Benefits include improving data quality of area-based surveys.,The system outputs predictions of the types of crops in specific locations within the Conterminous United States (CONUS).,Operation and Maintenance,Neither,1/1/2021,1/1/2021,2/15/2021,,Developed in-house.,,No,,,No,,No,No,Yes,,The data is derived from Cropland Data Layer (CDL) and Farm Service Agency (FSA) administrative data. CDL is produced using satellite images and extensive agricultural ground reference data.,"Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,6-12 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Dam Inspection Report Document Processing,Department of Agriculture,USDA,NRCS: Natural Resources Conservation Service,Mission-Enabling,,None of the above.,"The purpose of this AI is to pull out and organize data from thousands of dam inspection documents so that we can use Microsoft Power BI to understand the condition of thousands of USDA Watershed program dams. This allows us to identify the biggest issues and trends across our collection of over 2,100 dams in Oklahoma while reducing labor hours required to complete the task manually.","The model outputs text and checkbox responses, including dam metadata, inspection issue tracking (yes and no checkboxes), and further remarks on the issue or what has been/needs to be done on the dam.",Operation and Maintenance,Neither,5/1/2023,9/1/2023,12/1/2023,,Developed in-house.,,No,,,No,,No,No,Yes,,"The data is made up of dam inspection forms collected by dam owners (ex. conservation districts, cities) or Natural Resources Conservation Service (NRCS) staff. The data was validated through manual checks and is composed of 30+ dam inspection reports with different handwriting. The data contains 120+ variables that are read from each two-page inspection report. The data types are date, text, table (text variable types in the table), and checkbox binary outputs (checked: yes or no). The data is not publicly available.","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,Yes,AI Builder,Less than 6 months,Other,Worked with other NRCS leadership to get this model AI credits again for use once Microsoft AI Builder was reapproved.,Yes,,Yes,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Portfolio Approval and Management (PAM) Bot,Department of Agriculture,USDA,ERS: Economic Research Service,Mission-Enabling,,None of the above.,"The purpose of the model is to improve the Economic Research Service (ERS) research approval process. The system reduces the time it takes to fill out information and seeking approval, improves information accuracy, and brings visibility to the approval status across various division functions.","The system provides three outputs: the approval status, summary recommendations, and generated citations.",Initiated,Neither,5/1/2024,,,,,,,,,,,,No,Yes,,,,,,,,,,,,,,,,,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, GIS Invasive Tree Extraction for Field Level Users,Department of Agriculture,USDA,NRCS: Natural Resources Conservation Service,Energy & the Environment,,None of the above.,"The puspose of the model is to estimate of the spread of invasive tree infestation, specifically Eastern redcedar. This helps to avoid poor or inaccurate estimates caused by time constraints and heavy workloads when manually collecting the data.",The model outputs polygons representing the extent of trees present in landscape.,Operation and Maintenance,Neither,7/15/2019,7/15/2019,9/30/2019,,Developed in-house.,,No,,,No,,No,No,Yes,,"The data is created from the National Agriculture Imagery Program (NAIP), aerial images of land taken from aircrafts. The data is not limited to NAIP, but the 2020 NAIP has proven to be the most reliable source data.","Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,No,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, DISTRIB-II: Habitat Suitability of Eastern United States Tree,Department of Agriculture,USDA,FS: Forest Service,Science & Space,,None of the above.,"The purpose and expected benefits of the Climate Change Atlas are to give forest resource managers, forest landowners, and the general public information on the current and potential future of habitats for various tree species in the eastern United States. This information can contribute to forest management decisions when considering how climate change may affect the trees currently present and how likely it is that other tree species not currently in an area might find new habitats under different climate change scenarios.","The system outputs predictions of how well a tree species can live in a certain habitat based on climate change scenarios. Maps, graphs, and reports are generated from the modeled geographic information systems (GIS) data.",Operation and Maintenance,Neither,4/10/1998,5/16/2016,9/12/2019,,Developed in-house.,,No,,,Yes,"The input data and predictions have been evaluated using standard statistical methods, reviewed by several experts and many users, and the project has been published in several peer-reviewed journals.",No,No,Yes,,The data is created from publicly available Forest Inventory and Analysis data.,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, FSA FLP Chatbot,Department of Agriculture,USDA,FSA: Farm Service Agency,Mission-Enabling,,None of the above.,The purpose of this use case is to solve the problem of searching Loan handbooks to provide better customer service. The expected benefit is to help employees provide better service. A second benefit that is being explored is providing Veteran specific answers to services.,The expected output is text answers to prompt questions. ,Acquisition and/or Development,Neither,12/14/2023,3/29/2024,,,Developed in-house.,,No,,,,,,No,Yes,,"The data is collected from the 1 and 3 Farm Loan handbooks that are available on USDA intranet. The handbooks contain text, tables, and charts. There are also images such as screen shots in the handbooks.","Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",Other,Veteran,,,,No,,6-12 months,Other,Using DISC Incubator,No,,No,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, ROE Document Recognition - RoeDR,Department of Agriculture,USDA,RMA: Risk Management Agency,Mission-Enabling,,None of the above.,"The purpose of this model is to analyze documents from producers and Authorized Insurance Providers (AIPs), pick the appropriate page from the documents, read the signature date and producer signature name, convert the date and name to text, and load it into an application. This feature saves us time from having to input the data manually. We can then use the data for reporting purposes.",The model outputs the producer signature and signature date within document as text.,Operation and Maintenance,Neither,5/15/2024,5/23/2024,6/12/2024,,Developed in-house.,,No,,,No,,No,No,Yes,,The data is created from actuarial change forms and determined yield requests received from authorized insurance providers. The data contained 150 documents. ,"Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",,,No,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,No,,No,,,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, IRIS,Department of Agriculture,USDA,APHIS: Animal and Plant Health Inspection Service,Science & Space,,None of the above.,The purpose of this system is to make literature searches more effective for Biotechnology Regulatory Services. This increases work efficiency with our regulatory tasks.,The model outputs recommended literature list for scientists.,Operation and Maintenance,Neither,1/9/2023,2/5/2024,5/15/2024,,Developed with contracting resources.,12639524P0188 ,No,,,No,,No,No,Yes,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,No,,No,,No,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Ticket Resolution Categorization (Incident/Change),Department of Agriculture,USDA,OCIO: Office of the Chief Information Officer,Mission-Enabling,,None of the above.,"The purpose of this model is to classify the resolution type and tier of all support desk tickets after they have been closed. This model allows the support team to spend more time identifying process inefficiencies and plan solutions rather than categorizing tickets. This will lead to process improvement, automation of repetitive tasks, increased productivity, and higher performance.",The model outputs the classification category of support ticket resolutions.,Operation and Maintenance,Neither,6/1/2023,6/1/2023,6/1/2023,,Developed with both contracting and in-house resources.,ITSS Contracts,No,,,No,,No,No,Yes,,"The dataset is composed of Remedy data, containing inventory data of all of assets at DISC: hardware, software, incident tickets, and change tickets. Data contains numerical, categorical, text, and date data types.","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Other,"SD105, disc employee or customer",Yes,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Ticket Templatization,Department of Agriculture,USDA,OCIO: Office of the Chief Information Officer,Mission-Enabling,,None of the above.,"This model is a non-production exploratory model, meaning it does not make predictions but is used to explore trends within data to gain insights that can help in making data-driven decisions. It is designed to explore and analyze service and change requests submitted through the 105 general form or without templates. This model helps to identify subcategories within the larger dataset that could be candidates for standardization and automation, potentially leading to improved operational efficiency, cost savings, and customer satisfaction.",The model outputs trends within data to assist in decision making.,Acquisition and/or Development,Neither,1/1/2024,1/1/2024,,,Developed with contracting resources.,ITSS Contracts,No,,,,,,No,Yes,,"The dataset is composed of Remedy data, containing inventory data of all of assets at DISC: hardware, software, incident tickets, and change tickets. Data contains numerical, categorical, text, and date data types.","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,,,,No,,Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, File Rename Automation,Department of Agriculture,USDA,FSA: Farm Service Agency,Mission-Enabling,,None of the above.,The purpose of this tool is to rename thousands of documents converted from physical to digital records that were given a generic file name. This tool can grab text from page 1 of each document and apply a correct file rename instead of employees having to spend hours manually renaming documents.,The model outputs renamed files.,Implementation and Assessment,Neither,11/6/2023,11/6/2023,,,Developed in-house.,,No,,,No,,No,No,Yes,,The data is created from agency records and were used to train Microsoft Power Automate to extract data from records and use in the file renaming process.,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,No,"Yes – agency has access to source code, but it is not public.",,No,,6-12 months,Yes,,No,,Other,Sometimes,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Rapid Drafting of ARS Research Summaries,Department of Agriculture,USDA,ARS: Agricultural Research Service,Mission-Enabling,,None of the above.,"The purpose of this tool is to summarize ongoing research from internal Agricultural Research Service (ARS) documents to allow program staff to quickly create accurate and timely summary documents, such as briefing papers, talking points for leadership, and speeches. This will give staff more time for other duties, and leadership will be able to confidently answer questions, justify budget requests, and ensure that our research is innovative and relevant.",The tool outputs talking points and short briefing papers.,Initiated,Neither,3/18/2024,,,,,,,,,,,,No,Yes,,,,,,,,,,,,,,,,,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, DS Hub Geo-metadata generation,Department of Agriculture,USDA,NRCS: Natural Resources Conservation Service,Government Services (includes Benefits and Service Delivery),,None of the above.,"The purpose of this AI use case is to generate metadata for Natural Resources Conservation Service (NRCS) datasets, ensuring consistency, accessibility, and compliance through generative AI. Expected benefits include increased data accessibility, reduced manual workload, minimized errors, better dataset understanding, and fast data retrieval for stakeholders.","The model outputs accurate, consistent, and compliant metadata appropriate for the existing geospatial data that it analyzed.",Initiated,Neither,9/20/2024,,,,,,,,,,,,No,Yes,,,,,,,,,,,,,,,,,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Dynamic Soils Hub ,Department of Agriculture,USDA,NRCS: Natural Resources Conservation Service,Science & Space,,None of the above.,"The Dynamic Soils Hub (DS Hub) under the Natural Resources Conservation Service (NRCS) is a tool designed to help both government workers and the public understand and analyze soil information. The DS Hub links different soil and conservation databases, making it easier to evaluate the environmental benefits of conservation programs by accessing previously separate data and models. This enhances the USDA’s ability to study and report on how soil properties change with conservation efforts over time.",The system outputs the class of soil based on the supplied soil information.,Implementation and Assessment,Neither,11/11/2020,11/11/2020,,,Developed with contracting resources.,,No,,,No,,No,No,Yes,,,"Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",,,No,"Yes – agency has access to source code, but it is not public.",,Yes,Dynamic Soils Hub ,6-12 months,Yes,,Yes,,Yes,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Cover Crop Mapping,Department of Agriculture,USDA,RMA: Risk Management Agency,Law & Justice,,None of the above.,"This project aims to measure the use of cover crops on farms in the U.S. Midwest by creating yearly maps showing cover crops in the fall and spring. These maps are made using satellite images and models of plant growth. This data helps the agency independently check and track how many farmers are using cover crops, reducing the need for on-site visits to determine if cover crops are present on a field.","The output is a state-level map of detected cover crops by year, classified by planting date (fall, spring).",Acquisition and/or Development,"Rights-Impacting ",9/2/2022,9/2/2022,,,Developed with both contracting and in-house resources.,This work is supported under a Research Support Agreement with the University of Illinois (RMA22CPT0012747),No,,,,,,No,Yes,,The data is created from satellite imagery that uses crop insurance cover crop subsidy data as an independent data validation source. Data is not publicly available.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,,,,No,,Less than 6 months,No,,No,,Other,N/A,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,Planned or in-progress.,"Benchmark evaluation: Testing of the AI model has involved the use of benchmarks (either publicly available or internally created) to estimate performance in real-world settings, but has not been tested in an operational environment. ","The current key risks for the Cover Crop detection are Type I and Type II errors. For this particular use case, a False Positive (Type I error) would be where cover crop was detected by the model, when in fact none was planted by the producer and False Negatives (Type II errors) where cover crop was planted and emerged, but the model did not detect cover crop. Model results will not be used without independent review before a potential compliance action takes place.",Planned or in-progress,No monitoring protocols have been established: Necessary infrastructure needed to perform monitoring of AI systems in production is not available and/or a plan to monitor models has not yet been established.,No - Some individual decisions or actions require direct human oversight.,,N/A - individuals are not interacting with the AI for this use case,,No,No analysis specific to this use case has been conducted as the model is focused on the detection of a particular land use-land cover class.,Direct user testing,,Yes,,"No – This AI use case is not subject to the opt-out requirement because the AI functionality is solely used for the prevention, detection, and investigation of fraud or cybersecurity incidents, or the conduct of a criminal investigation.", Planting Date Detection,Department of Agriculture,USDA,RMA: Risk Management Agency,Law & Justice,,None of the above.,"This project aims to find out the planting dates for corn, soybean, and winter wheat on farms in the U.S. Midwest. Maps containing planting dates are made using satellite images and models of plant growth. This data helps the agency independently check and confirm the planting dates reported by farmers to ensure the integrity of their programs, reducing the need for on-site visits to determine when a field was planted.","The output is an annual map of planting dates for corn, soybean, and winter wheat for crop years 2016-2023.",Acquisition and/or Development,"Rights-Impacting ",9/7/2022,9/1/2023,,,Developed with both contracting and in-house resources.,This work was developed under a Research Support Agreement with the University of Illinois (RMA22CPT0012747),No,,,,,,No,Yes,,The data is created from satellite imagery that uses crop insurance unit-level planting dates at the crop/type/practice level as an independent data validation source. Data is not publicly available.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,,,,No,,Less than 6 months,,,,,,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,Planned or in-progress.,"Benchmark evaluation: Testing of the AI model has involved the use of benchmarks (either publicly available or internally created) to estimate performance in real-world settings, but has not been tested in an operational environment. ","The current key risks for the planting date model results are Type I and Type II errors. For this particular use case, a False Positive (Type I error) would be where the planting date detected by the model was outside the margin of error, when in fact the planting date occurred within the margin of error and False Negatives (Type II errors) where the planting date detected by the model was inside the margin of error, when in fact the field was planted either earlier or later. Model results will not be used without independent review before a potential compliance action takes place. The model produces a crop-field specific observation (planting date) and is not tied to a demographic group.",Planned or in-progress,"Automated and Regularly Scheduled Updates: Some aspects of the plan to monitor the AI system post-deployment are automated including re-training of models after detecting drift; however, data science teams are still significantly involved in the monitoring and re-deployment process.",No - Some individual decisions or actions require direct human oversight.,,N/A - individuals are not interacting with the AI for this use case,,Yes,The model produces a crop-field specific observation (planting date) and is not tied to a demographic group.,Direct user testing,,"No – Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.",Models are currently in development & broader plans about the use of the results are happening internally.,"No – This AI use case is not subject to the opt-out requirement because the AI functionality is solely used for the prevention, detection, and investigation of fraud or cybersecurity incidents, or the conduct of a criminal investigation.", Acreage and Crop Type Validation,Department of Agriculture,USDA,RMA: Risk Management Agency,Law & Justice,,None of the above.,"This project uses satellite images and plant growth models to check if the reported sizes of farm fields and the types of crops grown match what is actually seen on the ground in the U.S. Midwest. This data helps the agency independently verify the accuracy of reported field sizes and crop types, supporting efforts to ensure the integrity of their programs.","The output is a validation of reported acreage and validation of reported crop type for corn, soybean, and winter wheat on farm fields.",Acquisition and/or Development,"Rights-Impacting ",9/1/2022,9/1/2022,,,Developed with both contracting and in-house resources.,This work was developed under a Research Support Agreement with the University of Illinois (RMA22CPT0012747),No,,,,,,No,Yes,,"The data is created from satellite imagery that uses planted acreage and crop type (corn, soybean, and winter wheat) features as an independent data validation source. Data is not publicly available.","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,,,,No,,Less than 6 months,,,,,,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,Planned or in-progress.,"Benchmark evaluation: Testing of the AI model has involved the use of benchmarks (either publicly available or internally created) to estimate performance in real-world settings, but has not been tested in an operational environment. ","The current key risks for the planting date model results are Type I and Type II errors. For this particular use case, a False Positive (Type I error) would be where the acreage detected by the model was outside the margin of error, when in fact the acreage reported was within the margin of error and False Negatives (Type II errors) where the acreage detected by the model was inside the margin of error, when in fact the acreage was outside the margin of error. Similar logic would be applied to the crop type detection. Model results will not be used without independent review before a potential compliance action takes place. The model produces a field specific observation and is not tied to a demographic group.",Planned or in-progress,"Automated and Regularly Scheduled Updates: Some aspects of the plan to monitor the AI system post-deployment are automated including re-training of models after detecting drift; however, data science teams are still significantly involved in the monitoring and re-deployment process.",No - Some individual decisions or actions require direct human oversight.,,N/A - individuals are not interacting with the AI for this use case,,Yes,No analysis specific to this use case has been conducted as the model is focused on the detection of a particular land use-land cover class and agronomic variable.,None of the above,,"No – Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.",Models are currently in development & broader plans about the use of the results are happening internally.,"No – This AI use case is not subject to the opt-out requirement because the AI functionality is solely used for the prevention, detection, and investigation of fraud or cybersecurity incidents, or the conduct of a criminal investigation.", U.S. Poultry Operations and Populations Dataset,Department of Agriculture,USDA,APHIS: Animal and Plant Health Inspection Service,Emergency Management,,None of the above.,"The purpose of this case is to develop a dataset that addresses the problem of not having complete information about where poultry farms are located and how many birds they have. Filling this gap provides detailed data on poultry farm locations and populations, which is essential for planning animal health emergencies and predicting the spread of diseases.",Output is a national-level dataset of domestic poultry operations and estimated populations. ,Operation and Maintenance,Neither,5/15/2016,5/15/2016,6/3/2019,,Developed in-house.,,No,,,No,,No,No,Yes,,The data consists of a subset of farm locations that were validated using manually digitized poultry operations in 41 counties. Data is not publicly available.,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,No,,Other,Used existing lab machines and enterprise licenses with Esri(GIS),,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Equine Operations and Populations Dataset for the U.S.,Department of Agriculture,USDA,APHIS: Animal and Plant Health Inspection Service,Emergency Management,,None of the above.,"The purpose of this case is to develop a dataset that addresses the problem of not having complete information about where horse farms are located and how many horses they have. Filling this gap provides detailed data on horse farm locations and populations, which is essential for planning emergencies and predicting the spread of diseases.",Output is a national-level dataset of domestic horse operations and estimated populations.,Acquisition and/or Development,Neither,1/2/2023,9/1/2024,,,Developed with both contracting and in-house resources.,"NADPRP / Farm Bill funds , cooperative agreement with Colorado State University",No,,,,,,No,Yes,,"The data is created from National Agriculture Imagery Program (NAIP) images and the National Agricultural Statistics Service Census of Agriculture, which includes number of farms by size and type, inventory and values for crops and livestock, producer characteristics, and much more. Data is not publicly available.","Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,,,,No,,Less than 6 months,No,,Other,In process with MRP GIS Coordinator,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, NASS - Naggle 2.0 Automated Editing Tool,Department of Agriculture,USDA,NASS: National Agricultural Statistics Service,Other,Agricultural Statistics,None of the above.,"The purpose of this model is to determine if an answer on a survey is valid or invalid. If an answer is classified as invalid, a regression model will then suggest a corrected value. This approach will help reduce errors and improve the accuracy of survey forms, saving time and reducing the number of labor hours spent on editing.","The classification model outputs an excel sheet with the survey, person, variable, and whether the variable is valid or invalid. The regression model outputs an excel sheet containing the invalid records, which includes the survey, person, variable, original value, and new predicted value. ",Initiated,Neither,6/3/2024,,,,,,,,,,,,No,Yes,,,,,,,,,,,,,,,,,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, County-level remotely-sensed corn and soybean yield estimation,Department of Agriculture,USDA,NASS: National Agricultural Statistics Service,Other,Agricultural Statistics,None of the above.,The purpose of this tool is to estimate yearly corn and soybean yields for each county using satellite images. More details can be found in the paper titled “An assessment of pre- and within-season remotely sensed variables for forecasting corn and soybean yields in the United States” (https://doi.org/10.1016/j.rse.2013.10.027). Benefits of providing county-level crop yield statistics allow stakeholders to make more informed planning and decisions.,The model outputs county-level crop yield estimates for corn and soybeans in the amount of bushels per acre.,Operation and Maintenance,Neither,1/1/2007,1/1/2009,1/1/2014,,Developed in-house.,,No,,,No,,No,No,Yes,,Historical National Agricultural Statistics Service county-level crop yields are used as the dataset. The foundation of the data is images from NASA Moderate Resolution Imaging Spectroradiometer (MODIS) satellites.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,No,"Yes – agency has access to source code, but it is not public.",,No,,More than 12 months,Yes,,No,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, NASSportal Intranet Bot,Department of Agriculture,USDA,NASS: National Agricultural Statistics Service,Mission-Enabling,,Searching for information using AI.,This model will assist National Agricultural Statistics Service (NASS) staff in finding answers to questions on how to administer programs. This will decrease labor hours and increase efficiency of NASS staff in program administration.,The chatbot will provide text outputs.,Initiated,Neither,7/1/2024,,,,,,,,,,,,No,Yes,,,,,,,,,,,,,,,,,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, XyloTron/XyloPhone Wood Identification System,Department of Agriculture,USDA,FS: Forest Service,Law & Justice,,None of the above.,The purpose of these tools is to identify different types of wood based on their cross-section. These tools will help industries follow laws and support law enforcement in meeting national (e.g. Lacey Act) and international (e.g. CITES) regulations.,The tools will output a prediction of the type wood.,Implementation and Assessment,Neither,1/1/2016,1/1/2016,,,Developed in-house.,,No,,,No,,No,No,Yes,,"The data contains images of the cross-section of polished wood surfaces. As of July 2024, there are 139,639 images of 10,888 wood specimens.","Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,No,,No,,,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Incident Invoice Document Understanding,Department of Agriculture,USDA,FS: Forest Service,Mission-Enabling,,None of the above.,The purpose of this tool is to analyze incident invoices and return the values that need to be entered into a database. This new approach leads to faster invoice processing and reduces data entry mistakes for more accurate data.,The tool outputs an Excel document containing required values identified from incident invoices.,Acquisition and/or Development,Neither,1/21/2021,1/26/2021,,,Developed with contracting resources.,12314419F0612 and another USDA DISC contract that I don't have access to,No,,,,,,No,Yes,,"The dataset is created with PDF files of OF-286 documents, which are paid invoices of varying quality and structure.","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,,,,Yes,UiPath,More than 12 months,Other,We are working closely with DISC to build the necessary infrastructure. It has been a moving target.,No,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Forest disease detection and screening,Department of Agriculture,USDA,FS: Forest Service,Science & Space,,None of the above.,"The purpose of this project is to improve tree disease diagnosis and screening, thereby facilitating ongoing efforts within and outside the Forest Service to manage diseases of forest trees. ","The model will output a prediction indicating whether a tree is diseased or not, and if a tree is resistant or susceptible to a disease.",Acquisition and/or Development,Neither,8/3/2020,8/3/2020,,,Developed in-house.,,No,,,,,,No,Yes,,"Data has and is currently being collected using commercially available and custom sensors and images from trees in natural forest landscape, trees in planted settings, and trees that are being screened for disease resistance in controlled environments. Data is collected for both model calibration and validation. ","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,,,,No,,More than 12 months,No,,Other,Use government furnished equipment,,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Use of LLMs for data extraction,Department of Agriculture,USDA,FS: Forest Service,Mission-Enabling,,None of the above.,"The purpose of this model is to quickly gather information from scientific papers to track plant diseases. The practical benefit is that using this method would save time compared to manually collecting the information, which can be slow and error-prone when done for a long time.","The model outputs a table of requested data variables (e.g., country, pathogen name, host name, etc.).",Initiated,Neither,10/1/2023,,,,,,,,,,,,No,Yes,,,,,,,,,,,,,,,,,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, IPWG Application Survey Analysis,Department of Agriculture,USDA,FS: Forest Service,Mission-Enabling,,None of the above.,"The purpose of this tool is to analyze over 5,000 responses from an internal employee survey on IT applications, then give a summary of the employee feedback regarding each IT application. This decreases the time required to go through each response manually, helping the team make informed investment decisions more quickly.","The model outputs a text summarization for each IT application in the survey data, and potentially includes text summaries of responses and sentiment analysis. The project will also produce a dashboard that allows users to see similar attributes at the agency, office, application, and individual response level.",Acquisition and/or Development,Neither,10/3/2024,10/3/2024,,,Developed in-house.,,No,,,,,,No,Yes,,"The data has been collected from a Microsoft Forms survey sent to Forest Service employees and downloaded into a CSV (comma-separated values) file. The survey responses are not anonymous and contain personnel information such as region, office, and job title. There are approximately 5000 responses about hundreds of applications. The data is comprised of approximately 54 attributes, with a mix of numerical ratings, categories, and free text comments. The data is restricted and encrypted due to personnel information. ","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,,,,Yes,"Excel, Python, Google Cloud Platform ",Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Fire Resilent Landscapes,Department of Agriculture,USDA,FS: Forest Service,Science & Space,,None of the above.,The goal of this tool is to quantify the cost of forest treatments. Benefits include providing the ability to accurately map treatment costs for users to make more informed decisions.,The tool outputs predictions in the form of raster surfaces/maps.,Implementation and Assessment,Neither,8/1/2021,8/1/2021,,,Developed in-house.,,No,,,Yes,,No,No,Yes,,No agency-owned data used in this project.,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,Yes,Yes – source code is publicly available.,,No,,Less than 6 months,No,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, PC Rasterize,Department of Agriculture,USDA,FS: Forest Service,Science & Space,,None of the above.,The purpose of this tool is to be able to process point cloud data more efficiently. This will reduce costs associated with processing point cloud data.,The tool outputs point clouds and raster surfaces/maps.,Implementation and Assessment,Neither,8/1/2024,8/1/2024,,,Developed in-house.,,No,,,No,,No,No,Yes,,No agency-owned data used in this project.,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,Yes,Yes – source code is publicly available.,,No,,Less than 6 months,No,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Spread and Balance Sample Design,Department of Agriculture,USDA,FS: Forest Service,Science & Space,,None of the above.,The purpose of this tool is to produce samples that are well spread and balanced. This sample design will reduce the quantity of samples needed and further reduce costs associated with collecting field data.,The tool outputs data frames and geospatial-data-frames.,Implementation and Assessment,Neither,5/1/2024,5/1/2024,,,Developed in-house.,,No,,,Yes,,No,No,Yes,,No agency-owned data used in this project.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,Yes – source code is publicly available.,,No,,Less than 6 months,No,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, "Regression, Classification, Clustering with Hilbert Curves",Department of Agriculture,USDA,FS: Forest Service,Science & Space,,None of the above.,"The purpose of this tool is to perform better regression, classification, and clustering. This will create a new and better ways to produce various estimates, reducing cost and error. ",The tools will output Data frame and Raster Surfaces/maps. ,Implementation and Assessment,Neither,6/1/2024,6/1/2024,,,Developed in-house.,,No,,,No,,No,No,Yes,,No agency-owned data used in this project.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,Yes – source code is publicly available.,,No,,Less than 6 months,No,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, "The Big Data, Mapping, and Analytics Platform (BIGMAP) Project",Department of Agriculture,USDA,FS: Forest Service,Science & Space,,None of the above.,The purpose of this project is to use geospatial predictions from Forest Inventory and Analysis samples to make more accurate estimates of different forest characteristics. Greater precision in estimates leads to more informed decisions about the forest resources in the US.,The model outputs predictions in the form of raster maps.,Operation and Maintenance,Neither,1/1/2019,1/7/2019,7/5/2021,,Developed with both contracting and in-house resources.,Esri FS GeoPlatform,No,,,Yes,The methods used are all documented in previously published peer-reviewed articles. Results are validated using data from the Forest Inventory and Analysis program and a peer-reviewed comparative assessment methodology.,Yes,No,Yes,,"Data from the Forest Inventory and Analysis program and the Forest Inventory and Analysis Database were used to finetune and validate the models used. While the data are publicly available, the actual coordinates of the plot locations used are not, in order to protect the confidentiality of the land owner.","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Esri Managed Cloud Services; Amazon Web Services,More than 12 months,Yes,,Yes,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, BirdNET to detect bird vocalizations for research and species monitoring,Department of Agriculture,USDA,FS: Forest Service,Science & Space,,None of the above.,"BirdNET quickly scans thousands of hours of forest audio recordings to detect bird calls from species that are important for forest monitoring, like spotted owls, black-backed woodpeckers, and willow flycatchers. This decreases the time and cost associated with manually listening to recordings to identify bird calls.","The model outputs text files of bird calls, which include the bird species and time that the call was recorded.",Operation and Maintenance,Neither,6/1/2021,6/1/2021,2/13/2023,,Developed in-house.,,No,,,No,,No,No,Yes,,BirdNET was developed using data from the MacAulay Library of Natural Sounds at Cornell University. Forest Service did not provide any contract or data. BirdNET is publicly available.,"Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",,,No,Yes – source code is publicly available.,https://www.sciencedirect.com/science/article/pii/S1574954121000273,No,,More than 12 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Hurricane impact descriptions,Department of Agriculture,USDA,FS: Forest Service,Mission-Enabling,,None of the above.,"The purpose of the AI model is to convert a table of data about a tropical cyclone's path and estimated impact on forests into a clear and understandable story. This is part of a rapid assessment given to stakeholders after a cyclone hits, so it needs to be done fast. We are creating a tool to automate this process and the AI helps to make better quality reports.",The model outputs a few paragraphs of easy-to-read text that explains the effects of a cyclone. ,Acquisition and/or Development,Neither,7/1/2024,7/1/2024,,,Developed in-house.,,No,,,,,,No,Yes,,"No agency owned data is used to train, fine-tune, or evaluate the performance. ","Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,,,,No,,Less than 6 months,,,Other,Not sure,Other,No communication necessary thus far,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Predictive flood modeling,Department of Agriculture,USDA,FS: Forest Service; USDOT: US Department of Transportation,Transportation,,None of the above.,The purpose of this tool is to predict water flow during floods and assesses the vulnerability of drains under roads. This will help U.S. Department of Transportation (USDOT) and USDA Forest Service to make informed decisions in drain restoration and protection against flooding.,The model outputs water flow predictions during flood events and the vulnerability level of drains under roads.,Initiated,Neither,10/1/2024,,,,,,,,,,,,No,Yes,,,,,,,,,,,,,,,,,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, FuelCast,Department of Agriculture,USDA,FS: Forest Service,Emergency Management,,None of the above.,"This project predicts future fuel conditions and gives early warnings to help plan fuel management. The benefits include better preparation of the US firefighting teams for potential increases in large wildfires. This system also reduces the workload for fire behavior analysts because it provides fuel estimates, so they don't have to spend as much time figuring out fire behavior patterns through trial and error.",The model outputs predictions of the future quantity of wood and plants that could be present and contribute to wildfires. ,Operation and Maintenance,Neither,10/17/2019,10/17/2020,10/18/2020,,Developed with both contracting and in-house resources.,I have no idea what PIID is,No,,,Yes,,No,No,Yes,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,More than 12 months,,,Other,Not sure,No,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, R1 Forest Vegetation Modeling,Department of Agriculture,USDA,FS: Forest Service,Energy & the Environment,,None of the above.,The purpose of this tool is to use satellite images and methods such as LiDAR (light detection and ranging) with machine learning to model forest vegetation and make estimates. The use of machine learning improves models and estimates with decreased time and cost.,The model outputs predictions of forests and vegetation in the form of raster and vector geospatial maps. ,Initiated,Neither,1/1/2024,,,,,,,,,,,,No,Yes,,,,,,,,,,,,,,,,,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, ESRI Support Chat Bot,Department of Agriculture,USDA,FS: Forest Service,Science & Space,,None of the above.,The purpose of this chatbot is to help handle requests for support with geospatial software between our team and the software vendor. The benefits include saving time when dealing with Environmental Systems Research Institute (ESRI) support issues and reducing the number of specific ESRI support tickets that need to be sent to the Forest Service Geospatial Helpdesk or to ESRI through contract support services.,"The chatbot outputs support ticket entries, code snippets for queries, and text and links for support ideas and answers.",Initiated,Neither,10/21/2024,,,,,,,,,,,,No,Yes,,,,,,,,,,,,,,,,,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Wildlife deterrent system,Department of Agriculture,USDA,FS: Forest Service; Forest Service Research & Development,Science & Space,,None of the above.,"The purpose of the AI device is to keep coyotes out of a fenced area by blocking their entry through a gap in the fence, while still allowing other wildlife to pass through. The benefits include making ecological research possible that couldn't be done otherwise, and saving time by reducing the need to watch camera footage and manually control the fence.","The model performs video object detection of a coyotes, and arms an electrical barrier to prevent passage of the coyote.",Implementation and Assessment,Neither,8/26/2021,8/26/2021,,,Developed with contracting resources.,GS-35F-0207T,No,,,No,,No,No,Yes,,The data is composed of a few thousand photographs of coyotes collected from trail cameras. Images were used by the contractor to train the system to detect coyotes.,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,No,,No,,,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, The Lost Meadows Model,Department of Agriculture,USDA,FS: Forest Service,Energy & the Environment,,None of the above.,"The purpose of this model is to find out where meadows used to be and how often they appeared in order to understand their original state and their potential for restoration. The discovery of these areas increases the potential for meadow restoration, which can benefit biodiversity, wildfire management, carbon storage, and water storage.","The model outputs predictions of areas with meadow-like environmental conditions. The predicted areas include a mixture of existing but undocumented meadows, non-meadow lands that may have once been meadows, and meadow-like areas that may never have been a meadow. ",Operation and Maintenance,Neither,10/10/2022,10/10/2022,7/1/2023,,Developed in-house.,,No,,,Yes,,No,No,Yes,,"The data is a collection of 11,127 publicly available meadow polygons from the Sierra Nevada MultiSource Meadow Polygons Compilation, Version 2 (UC Davis and USDA Forest Service 2017). ","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,No,,No,,Other,No questions had,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Markov random fields,Department of Agriculture,USDA,FS: Forest Service,Science & Space,,None of the above.,The purpose of this tool is to improve the accuracy of estimates in machine learning models. The benefits include helping stakeholders make more informed and effective decisions for managing mixed forests.,"The model outputs predicted counts of tree species in a location, and the degree of competition between different tree species in the same location.",Acquisition and/or Development,Neither,10/1/2022,10/1/2022,,,Developed in-house.,,No,,,,,,No,Yes,,The data is comprised of ForestGEO tree count data from the Republic of Palau.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,,,,No,,More than 12 months,Yes,,Yes,,No,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, AI for regional forest mapping and monitoring,Department of Agriculture,USDA,FS: Forest Service,Energy & the Environment,,None of the above.,The purpose of this model is to use existing satellite images and forest survey data from the USDA Forest Service to create detailed maps of forest structures. This information will help land managers be more effective and efficient with their planning.,"The model outputs GeoTiffs (raster maps of forest attributes, such as tree density and tree species data).",Operation and Maintenance,Neither,1/1/2000,1/1/2000,10/1/2002,,Developed with contracting resources.,22-JV-11261959-067 (with Oregon State University; other past JVAs as well),No,,,Yes,Mapping products are currently reviewed by for the USDA Forest Service Forest Inventory and Analysis program and the USDA Forest Service Region 6 Biometrics Program. All products are distributed with detailed accuracy assessments.,No,No,Yes,,"Forest Inventory and Analysis data collected by the USDA Forest Service (Research & Development). It contains 200,000 individual plot measurements, and hundreds of Geospatial Information System (GIS) features (climate data, satellite images, topography, etc.) stored as 30-meter raster maps (GeoTiffs).","Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",,,Yes,Yes – source code is publicly available.,,No,,Less than 6 months,Other,"N/A, AI application performed by Oregon State University, not USDA Forest Service",Other,"N/A, AI application performed by Oregon State University, not USDA Forest Service",Other,"N/A, AI application performed by Oregon State University, not USDA Forest Service",Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, IOL Focus Group and Survey Sensemaking,Department of Agriculture,USDA,FS: Forest Service,Education & Workforce,,None of the above.,The purpose of this model is to efficiently process large quantities of focus group transcripts and survey results. Benefits include decreased labor hours manually processing transcripts and surveys.,The model outputs text summaries of focus group comments and surveys.,Initiated,Neither,6/11/2024,,,,,,,,,,,,No,Yes,,,,,,,,,,,,,,,,,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Esri ArcGIS Pro Deep Learning Modules,Department of Agriculture,USDA,FS: Forest Service,Mission-Enabling,,None of the above.,"The purpose of this tool is to enhance scientific modeling and analysis, which will standardize Geographic Information System (GIS) workflows for modeling and analytics.",The tools will output image classifications.,Operation and Maintenance,Neither,4/1/2024,4/1/2024,4/1/2024,,Developed in-house.,,No,,,No,,No,No,Yes,,The data is comprised of Natural Resource image data.,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,Yes,FS ACE,More than 12 months,Yes,,Yes,,,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, EMC Comment Parsing and Analysis,Department of Agriculture,USDA,FS: Forest Service,Government Services (includes Benefits and Service Delivery),,None of the above.,"This project aims to extract, categorize, and respond to public comments based on past responses. The benefits include creating a standardized process for handling comments, making public comment data more accessible and ready for AI use, reducing the time and cost of processing comments, minimizing human errors due to high workloads and tight deadlines, improving responsiveness to public concerns, increasing public trust, enhancing accountability through clear reporting, and supporting team training by building a database of common themes and response strategies.",The model will output text analyses of categories pulled from public comments and recommend responses based on historic responses.,Initiated,Neither,3/1/2024,,,,,,,,,,,,No,Yes,,,,,,,,,,,,,,,,,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, QUIC-Fire processing and analysis,Department of Agriculture,USDA,FS: Forest Service,Science & Space,,None of the above.,AI is being used to analyze data from fire-atmosphere models to understand fire behavior and effects. The goal is to create tools that will help fire and smoke managers use the QUIC-Fire (Quick Urban & Industrial Complex-Fire) model for planning controlled burns and assessing wildfire behavior.,The AI output will be a collection of metrics that provide building blocks for a tool that fire and smoke managers will use to implement QUIC-Fire (Quick Urban & Industrial Complex - Fire) into their decision-making.,Initiated,Neither,9/25/2024,,,,,,,,,,,,No,Yes,,,,,,,,,,,,,,,,,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Analysis of prescribed fire turbulence data,Department of Agriculture,USDA,FS: Forest Service,Science & Space,,None of the above.,The purpose of this project is to find connections between the heat from a wildfire and the turbulence it creates in the air. Current tools are not very accurate and can make mistakes. This AI effort helps create better tools that can assist fire and smoke managers in making decisions about smoke management.,The model outputs correlation analysis of how temperature change is associated with air turbulence measurements above a prescribed fire.,Acquisition and/or Development,Neither,6/12/2023,7/10/2023,,,Developed in-house.,,No,,,,,,No,Yes,,The data is comprised of temperature and turbulence kinetic energy readings collected over time from 16 sonic anemometers (measure wind speed and direction) and 112 thermocouples (measures temperature). The datasets are published in the US Forest Service data archive. ,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,,,,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Complaint Summarization,Consumer Financial Protection Bureau,CFPB,CFPB,Mission-Enabling,,None of the above.,Support staff that analyze complaints by helping to scale internal review of consumers' complaint narratives and companies' response narratives in light of increasing complaint volumes. Cost avoidance. ,Brief narrative summaries describing concepts included in complaint narratives. ,Initiated,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Ask Complaints,Consumer Financial Protection Bureau,CFPB,CFPB,Mission-Enabling,,None of the above.,Support staff that analyze complaints by helping to scale production of quantiative data in response to questions posed by users. Cost avoidance.,Summary data points of cohorts of complaints. ,Initiated,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Complaint Narrative Redaction,Consumer Financial Protection Bureau,CFPB,CFPB,Mission-Enabling,,None of the above.,"Redact personally identifiable information (PII) and other information that could be used to identify individuals from consumers' complaint narratives to support data sharing at scale. Cost containment, privacy protection, more informed consumer consent. ",Complaint narratives redacted of PII and other information that could be used to identify users to inform consumers' decisions about whether or not to publish their narratives. ,Initiated,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Document Review Tool,Consumer Financial Protection Bureau,CFPB,CFPB,Law & Justice,,None of the above.,Support investigations through document review assistance,Key documents,Initiated,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Anomaly Detection for Data Quality,Commodity Futures Trading Commission,CFTC,DOD,Mission-Enabling,,None of the above.,Anomaly detection model designed to identify potentially erroneous data loads in TCR data. Uses an isolation forest model with and automatically runs daily. ,Model provides an anomaly score on the full day of data for a single exchange,Operation and Maintenance,Neither,2/1/2023,5/1/2023,7/1/2023,,Developed with both contracting and in-house resources.,REDACTED,No,,,No,N/A,No,Yes,Other,Acquisition of the data catalog was concurrent with development of the use case ,Transactions by market and type,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,REDACTED,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Spoofing Detection AI/ML Project,Commodity Futures Trading Commission,CFTC,DOE,Law & Justice,,None of the above.,Pilot project that applied AI/ML techniques to identify spoofing patterns in order message data. Project used a variety of supervised and unsupervised techniques trained with DOE's existing set of expert based spoofing detection algorithms.,Model provides probability of spoofing behavior for every individual trader in a specified market on a specified day,Retired,Both,8/1/2023,9/1/2023,12/1/2023,4/26/2024,Developed with both contracting and in-house resources.,REDACTED,No,,,No,N/A,No,Yes,Other,Acquisition of the data catalog was concurrent with development of the use case ,Order message data,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",None,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,REDACTED,Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,No,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.",Risks inherent to any analytical identification: risk of false positives; risks identified by discussion among developers and use case owners,Yes – by another appropriate agency office that was not directly involved in the system’s development,No monitoring protocols have been established: Necessary infrastructure needed to perform monitoring of AI systems in production is not available and/or a plan to monitor models has not yet been established.,"Other – Immediate human intervention is not practicable; however, an appropriate fail-safe has been implemented. ","The model flags conduct for expert review. If the human reviewer were to see evidence of spoofing, the trader would be referred to the existing, human, pre-investigatory enforcement review process ",N/A - individuals are not interacting with the AI for this use case,,Yes,We follow the same procedures as we do for all enforcement matters.,None of the above,,"No – Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.","Traders have rights in any enforcement process, including any initiated by AI output.","No – This AI use case is not subject to the opt-out requirement because the AI functionality is solely used for the prevention, detection, and investigation of fraud or cybersecurity incidents, or the conduct of a criminal investigation.", Stress Testing Scenarios with Deep Learning,Commodity Futures Trading Commission,CFTC,DCR,Mission-Enabling,i,None of the above.,Pilot project to explore neural-network based machine learning methods for creating stress testing scenarios and estimating PnL (profit and loss) on FO (Futures and Options) portfolios based on the current/recent market states/conditions. ,The AI generates predicted price movements based on provided input probabilities ,Acquisition and/or Development,Neither,1/6/2023,2/14/2023,,,Developed with both contracting and in-house resources.,REDACTED,No,,,No,N/A,Yes,No,,,,,None,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, MPD Entity Risk Modeling,Commodity Futures Trading Commission,CFTC,MPD,Mission-Enabling,,None of the above.,Entity-level risk modeling project. Use statistical and probabalistic models to predict firms experiencing changes in capital levels. Very early stage R&D effort. ,Now-cast score for each registrant and each day indicating probability or risk category for near-future capital adjustment indicator statisitc,Initiated,Neither,1/1/2025,,,,Developed in-house.,,No,,,No,N/A,Yes,No,,,,,None,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Microsoft Copilot Integration,Election Assistance Commission,EAC,OCIO,Mission-Enabling,,Summarizing the key points of a lengthy report using AI.,Efficiency and improved administrative tasks.,Text-based summaries and AI generated text. ,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,No – Agency did not request an extension for this use case.,No,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.",AI is not safety or rights-impacting.,AI is not safety or rights-impacting.,AI is not safety or rights-impacting.,AI is not safety or rights-impacting.,,AI is not safety or rights-impacting.,,No,AI is not safety or rights-impacting.,None of the above,,No – it is not operationally practical to offer this.,AI is not safety or rights-impacting.,Other,"AI is not safety or rights-impacting, is not required for the completion of any tasks, and is optional for individual employees to use." Clearinghouse Network ,Election Assistance Commission,EAC,Clearinghouse,Government Services (includes Benefits and Service Delivery),,Summarizing the key points of a lengthy report using AI.,"The Clearinghouse Network is a series of online communities hosted on the Roundtable platform. These communities, including any AI features, are not open to the general public. Members of these communities are state or local election officials. The AI tools in this platform are intended to meet the needs of election officials by providing summaries for longer text or video based content. ",Summarized text and video transcripts.,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Language Translation,Equal Employment Opportunity Commission,EEOC,,Government Services (includes Benefits and Service Delivery),,Improving the quality of written communications using AI tools.,"Improve quality, efficiency and/or access to services",Generation. Text to speech. Speech to Text.,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Reasonable Accommodation,Equal Employment Opportunity Commission,EEOC,,Government Services (includes Benefits and Service Delivery),,Transcribing and summarizing a recorded meeting or interview using AI.,"Improve quality, efficiency and/or access to services",Generation. Text to speech. Speech to Text.,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Virtual Assistants,Equal Employment Opportunity Commission,EEOC,,Government Services (includes Benefits and Service Delivery),,Summarizing the key points of a lengthy report using AI.,"Improve quality, efficiency and/or access to services",Generation. Text to speech. Speech to Text.,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Employee Training and Professional Development,Equal Employment Opportunity Commission,EEOC,,Education & Workforce,,Collaborating in real-time using AI-assisted tools in word processors.,"Improve quality, efficiency and/or access to services",Generation. ,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Legal Research & Discovery,Equal Employment Opportunity Commission,EEOC,,Law & Justice,,Searching for information using AI.,"Improve quality, efficiency and/or access to services",Generation. Relevant legal information to support enforcement and/or litigation activity,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Records Management,Equal Employment Opportunity Commission,EEOC,,Mission-Enabling,,Inputting large amounts of data from paper forms into a digital system using AI.,Establish and maintain integrity of electronic data,"Generation. create digital documents or information that is organized, indexed, and accessed according to compliance standards.",Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, FOIA,Equal Employment Opportunity Commission,EEOC,,Mission-Enabling,,Prioritizing and categorizing incoming emails using AI.,Improve FOIA process,Generation. categorization/classification of priority or topic for policy or legal public comments,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Cybersecurity,Equal Employment Opportunity Commission,EEOC,,Mission-Enabling,,Identifying unusual patterns in system logs from a single incident report using AI.,Strengthen Cybersecurity practices,Anomalies. Classification.,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Invoice and Contract Data Extraction,Federal Deposit Insurance Corporation,FDIC,Chief Information Officer Organization,Mission-Enabling,,None of the above.,The purpose is to extract data from invoices and contracts (PDFs) and use the data for reconciliation. The benefit is to improve the accuracy of invoice and contract reconciliation and to reduce the time for Oversight Managers (OMs) to perform this activity.,"The system generates an excel output sent via email to OMs that show the results of the reconciliation, including discrepancies and errors.",Acquisition and/or Development,Neither,3/15/2024,4/15/2024,,,Developed with both contracting and in-house resources.,FDIC does not use PIIDs,No,,,,,,,Other,FDIC's Enterprise Data Catalog is still being implemented,Data from FDIC invoices and contracts,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",None,,,,,Yes,Commodities Invoice Reconciliation,Less than 6 months,Other,Various intake processes are available depending on the nature and scope of the request,Other,Various resource request processes are available depending on the nature and scope of the request,Other,Communication timeliness depends on the process relevant to the request. Many processes are automated and status updates are push instantaneous or are available on demand. Other manual processes involve the requestor at each step for transparency.,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Server Failure Prediction,Federal Deposit Insurance Corporation,FDIC,Chief Information Officer Organization,Mission-Enabling,,Identifying unusual patterns in system logs from a single incident report using AI.,"Server outages create work stoppages for FDIC. This use case will use predictive intelligence to predict when a server is about to fail, and will use natural language processing and query to create a help desk ticket for a server about to fail. ",One output will be the predictive intelligence list of servers likely to fail. A second output will be the system generated help desk ticket. ,Acquisition and/or Development,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Plain Language Policy Assistant,Federal Deposit Insurance Corporation,FDIC,Chief Information Officer Organization,Mission-Enabling,,Improving the quality of written communications using AI tools.,Policy writers at the FDIC need to ensure drafted policies comply with the Plain Language Writing Act. The system would be an AI that provided suggested feedback to the user's draft to better comply with the Plain Language Writing Act (PLWA),"The AI systems outputs would provide feedback on the evaluated the draft for clarity and simplicity, active voice, conciseness, jargon and acronyms. The AI would also redraft selected sections, with PLWA principles applied. ",Initiated,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Knowledge Article Generation,Federal Deposit Insurance Corporation,FDIC,Chief Information Officer Organization,Mission-Enabling,,Improving the quality of written communications using AI tools.,"The problem is that staff spends time writing article to resolve server outages. Following the remediation of a server outage incident, the Gen AI would write a draft Knowledge Article on how to resolve future incidents of that type. Other benefits: ~Providing a better service experience: The Infrastructure Operations Systems Branch (IOSB) team can leverage self-service; they can easily find answers; the IOSB service experience will improve, allowing IOSB team to self serve 24/7 from the Knowledge Articles base ~Reduce caseload on IOSB; IOSB can solve their problems more efficiently, fewer cases require IOSB support. IOSB freed up to solving more complex customer issues ~Cut down average handle time: As we solve server outage cases and structure into articles, more IOSB team members can use that information to solve their cases faster and reduce average handle time. ",The output would be a draft Knowledge Article.,Acquisition and/or Development,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, AI Coding Assistant,Federal Deposit Insurance Corporation,FDIC,Chief Information Officer Organization,Mission-Enabling,,None of the above.,"Enhance coding speed and accuracy by offering code suggestions in real time, writing unit tests, debugging code, explaining code, adding comments, and translating code across languages. ",Code suggestions and suggested comments to explain code,Initiated,Neither,4/18/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Content Creation,Federal Deposit Insurance Corporation,FDIC,Corporate University,Mission-Enabling,,None of the above.,"Reducing time to create content, increase volume of content, provide content for students who struggle to take notes and engage in class. ","Generative, Provides interactive resources used for training.",Initiated,Neither,11/4/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Immersive Learning,Federal Deposit Insurance Corporation,FDIC,Corporate University,Mission-Enabling,,None of the above.,"Immersive learning offers an engaging approach to training programs by creating realistic environments using GenAI where learners can practice and refine their skills. A controlled environment wherein learners interact with immersive technology to solve pre-determined complex situations and problems. This approach amplifies employee engagement and retention, performance management, learning and development, administrative tasks, workforce planning, employee onboarding, recruiting, interviewing as well as others. ",Prediction based on sequencing provided by developer; completions reports with objectives met.,Acquisition and/or Development,Neither,2/6/2024,11/22/2024,,,Developed with contracting resources.,FDIC does not use PIIDs,No,,,,,,,Other,FDIC's Enterprise Data Catalog is still being implemented,No specific FDIC datasets go into this model,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",None,,,,,No,,Less than 6 months,Other,Various intake processes are available depending on the nature and scope of the request,Other,Various resource request processes are available depending on the nature and scope of the request,Other,Communication timeliness depends on the process relevant to the request. Many processes are automated and status updates are push instantaneous or are available on demand. Other manual processes involve the requestor at each step for transparency.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Course Recommendation ,Federal Deposit Insurance Corporation,FDIC,Corporate University,Mission-Enabling,,None of the above.,"Have capability to recommend courses based on history/need and would consider it in the future. This approach amplifies employee engagement and retention, performance management, learning and development milestones, administrative Task, workforce planning, and employee onboarding. ","Recommendation, suggesting learning paths Provides industry specific training recommendations based previous training listed in your learning path. ",Initiated,Neither,7/22/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Student Monitoring ,Federal Deposit Insurance Corporation,FDIC,Corporate University,Mission-Enabling,,None of the above.,"To track course progress, comprehension and enable adaptive learning. Improve engagement, to provide performance management, workforce planning, success of employee on-boarding and encourage learning and development. ","Prediction and adaption analyzing and adjusting to support learning Provides completion rates, course milestones. ",Initiated,Neither,7/22/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Plagiarism Detection ,Federal Deposit Insurance Corporation,FDIC,Corporate University,Mission-Enabling,,None of the above.,"Detect and prevent cheating and dishonesty, Promoting the FDIC's values including Fairness, Accountability, and Integrity. Reduce the time manually inspecting for plagiarism. ","Report with flagged suspected plagiarism, and sources ",Initiated,Neither,7/22/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Student Outcome Projection ,Federal Deposit Insurance Corporation,FDIC,Corporate University,Mission-Enabling,,None of the above.,"Early intervention in the students academic growth providing the ability to support different learning styles and levels helping guide the right educational path forward personalizing learning Streamlining learning and development for industry and office needs allowing employees reduction of tasks for onboarding and workforce planning and retention. ",Prediction; Curriculum mapping based on job specific tasks that results in each user and provide position specific training report. ,Initiated,Neither,7/22/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Facilitating Status ,Federal Deposit Insurance Corporation,FDIC,Corporate University,Mission-Enabling,,None of the above.,"Monitor activities and administer evaluations in supervised and unsupervised environments, ensure compliance, and facilitate accreditation of course content allowing industry standards to be met by Corporate University (CU) courses. E.g. Accounting CEUs (Continuing Education Units).",Report that confirms physical attendance for in-person and virtual sessions.,Initiated,Neither,7/22/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, AI Assisted Data Collection,Federal Deposit Insurance Corporation,FDIC,Division of Depositor & Consumer Protection,Other,Financial Regulation,None of the above.,"The Assistant enhances the efficiency and effectiveness of our fair lending and compliance functions by automating data collection from PDF form documents to support internal supervisory consultations. Through using the Assistant. data collection takes significantly less time, retains and improves accuracy, and allows the FDIC examiners to focus on examination activities",The Assistant provides extracted data from credit reports and similar type documents in excel readable formats,Acquisition and/or Development,Neither,4/27/2023,4/27/2023,,,Developed in-house.,,No,,,,,,,Other,FDIC's Enterprise Data Catalog is still being implemented,This program uses bank-specific data from examinations containing bank-provided customer information from credit reports and loan applications often through FDICConnect,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,,,,Yes,Microsoft Power Apps - AI Builder,Less than 6 months,Other,Various intake processes are available depending on the nature and scope of the request,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, AI Assisted Time Management,Federal Deposit Insurance Corporation,FDIC,Division of Depositor & Consumer Protection,Other,Financial Regulation,Logging and analyzing time spent on tasks using AI-powered time management tools.,Increase efficiencies of using Process HQ through FOCUS (Framework for the Oversight of Compliance and CRA Activities User Suite) to understand where Examination process steps slow down,A report of time spent and identification of bottlenecks,Initiated,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Compliance Risk Monitoring,Federal Deposit Insurance Corporation,FDIC,Division of Depositor & Consumer Protection,Other,Financial Regulation,None of the above.,Enhance effectiveness of off-site risk monitoring on several regulation areas.,A report from Division of Consumer Protection (DCP) Analytics that identifies whether a financial institution has high / low relative risk of a compliance violation. Used by examiners to scope exams.,Acquisition and/or Development,Neither,8/14/2023,8/14/2023,,,Developed in-house.,,No,,,,,,,Other,FDIC's Enterprise Data Catalog is still being implemented,"Data comes from three major FDIC data sources, i.e. Research Information System database (RIS), compliance exam data (Framework for the Oversight of Compliance and CRA Activities User Suite ((FOCUS)) and consumer complaints data ((Enterprise Public Inquiries and Complaints ((EPIC)). ","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,,,,Yes,R,Less than 6 months,Other,Various intake processes are available depending on the nature and scope of the request,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, Analysis of Consumer Complaints,Federal Deposit Insurance Corporation,FDIC,Division of Depositor & Consumer Protection,Other,Financial Regulation,Searching for information using AI.,Identify consumer complaint trends and integrate pre-processing of consumer complaints using AI to enhance adjudication of consumer complaints,Cleaned up and categorized consumer complaints and trend analysis which does into data lake for post-processing analysis.,Initiated,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Pre-Examination Planning Monitoring,Federal Deposit Insurance Corporation,FDIC,Division of Depositor & Consumer Protection,Other,Financial Regulation,None of the above.,Assist examiners in pre-exam planning and identify pre-exam questions that are misaligned or do not inform risk assessments,A report from Division of Consumer Protection (DCP) Analytics to DCP Compliance and UDAP (Unfair Deceptive Acts and Practices) section related to pre-exam questions on their alignment with risk identification,Operation and Maintenance,Neither,2/8/2023,2/8/2023,12/14/2023,,Developed in-house.,,No,,,No,N/A,No,Yes,Other,FDIC's Enterprise Data Catalog is still being implemented,"PEP IR (Pre-Examination Planning Instrument Report) uses pre-exam planning documents such as ARCH (Assessment of Risk of Consumer Harm), Call Report data, to identify pre exam planning questions for further review.","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,R,6-12 months,Other,Various intake processes are available depending on the nature and scope of the request,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, Einstein Call Center Assistance,Federal Deposit Insurance Corporation,FDIC,Division of Depositor & Consumer Protection,Other,Financial Regulation,None of the above.,"Improve categorization and routing of consumer cases within Epic (Enterprise Public Inquiries and Complaints), to call agents and tag more effectively and efficient",Categorization and routing of calls and cases.,Initiated,Neither,4/19/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, HMDA (Home Mortgage Disclosure Act) Outlier Screen,Federal Deposit Insurance Corporation,FDIC,Division of Depositor & Consumer Protection,Other,Financial Regulation,None of the above.,"The HMDA (Home Mortgage Disclosure Act) Outlier program pre-screens off-site mortgage data to identify heightened fair lending risks related to pricing, underwriting, and redlining",Reports results of statistical analysis to support examination supervisory actions,Operation and Maintenance,Neither,1/1/2018,1/1/2018,1/1/2019,,Developed in-house.,,No,,,No,N/A,Yes,Yes,Other,FDIC's Enterprise Data Catalog is still being implemented,The HMDA Outlier program uses regulator version HMDA data provided by the CFPB. The public version of HMDA data strips out or obfuscates PII.,"Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.","Race/Ethnicity, Sex/Gender, Age",,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,SAS,Less than 6 months,Other,Various intake processes are available depending on the nature and scope of the request,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, FDIC Deposit Insurance Misrepresentation ,Federal Deposit Insurance Corporation,FDIC,Division of Depositor & Consumer Protection,Other,Financial Regulation,None of the above.,Division of Consumer Protection (DCP) Analytics is using AI to search social media posts to identify potential misrepresentation of FDIC deposit insurance,A spreadsheet report identifying specific social media posts related to a bank and FDIC deposit insurance ,Initiated,Neither,5/9/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Aggregating Complaint Data,Federal Deposit Insurance Corporation,FDIC,Division of Depositor & Consumer Protection,Other,Financial Regulation,None of the above.,DCP (Depositor & Consumer Protection) Exams has proposed to explore ways to increase efficiencies of analyzing complaint data to easily identify trends/concerns at our most complex banks.,To be determined,Initiated,Neither,11/20/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Financial Well-Being Project,Federal Deposit Insurance Corporation,FDIC,Division of Insurance & Research,Other,Financial Inclusion,None of the above.,(1) Developed a census of non-profit organizations that provide one or more of 9 types of financial well-being services. Such a census did not exist. (2) This census would advance the FDIC's focus on increasing economic inclusion by enabling DCP Community Affairs staff to more easily connect banks with the non-profits in their communities that provide specific types of financial well-being services.,Classification of the types of financial well-being services provided by a non-profit organization.,Retired,Neither,,,,12/31/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Generative Artificial Intelligence (AI) for Legal Research,Federal Deposit Insurance Corporation,FDIC,Division of Administration,Mission-Enabling,,Searching for information using AI.,This use case will allow users the ability to find the answers they need for legal research faster and easier. This directly supports our mission by providing users with enhanced research capabilities through generative AI. ,Commercially available legal research tools will provide users with generative AI capabilities to give attorney's information faster.,Initiated,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Outlook Unstructured Text Extraction ,Federal Deposit Insurance Corporation,FDIC,Division of Administration,Mission-Enabling,,None of the above.,The Corporate Services Branch (CSB) would like to use a custom AI prompt to test extraction from unstructured text emails that are currently going to a group box. This effort will be used to condition the data so that it can be used for reporting efforts and other needs. This is meant to be a pilot test to determine if this course of action is feasible and as a training opportunity for the Corporate Services Branch (CSB) Citizen Developer. Custom AI prompts are used for this use case.,The Corporate Services Branch (CSB) Pilot will automatically extract unstructured text from group email boxes when new email arrives to write the data into a usable data table for further use in management reporting and Key Performance Measures tracking. The model will also be used to look at historical data sitting in group mail boxes. ,Initiated,Neither,6/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Public-Facing ChatBot,Federal Deposit Insurance Corporation,FDIC,Division of Administration,Other,Financial Regulation,None of the above.,The chatbot is an Enterprise Information and records Management Section (EIRMS) effort. It will support the FDIC mission by mitigating calls to the FDIC Contact Center through the current RASA open-source platform. This project is waiting for a vendor platform to support their chatbot technology in the government cloud environment. ,The ChatBot will use a Natural Language Processer (NLP) to provide standard responses to consumers inquiring about deposit insurance and what is covered by the FDIC.,Initiated,Neither,7/1/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Purchase Card Activity Monitoring,Federal Deposit Insurance Corporation,FDIC,Division of Administration,Other,Financial Regulation,None of the above.,"The monitoring of prohibited and restricted purchase card information is a manually intensive process, whereas a Machine Learning (M/L) algorithm would have a level of complexity to determine whether purchases fit within those distinct set of parameters. This AI use cases would require the development of a training and test dataset based on actual activity in order to determine a true level of accuracy for the Machine Learning (M/L) algorithm. The data would then feed into a dashboard/report. ",The Machine Learning (M/L) would provide a data extract on prohibited and restricted purchase card activity across the FDIC.,Initiated,Neither,2/1/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, AI-driven Workflow Automations to Improve Business Operations,Federal Deposit Insurance Corporation,FDIC,Division of Administration,Mission-Enabling,,None of the above.,"Working with vendor software on some AI capabilities for use in the future.  There are various AI capabilities that may come available in a vendor's software (Generative AI to support human interactions, Intelligence chatbot, etc.) Human Resources (HR) is specifically looking at AI driven workflow automation to improve business operations.",The use case output will include decision trees to support real-time insights into business operations and help to identify trends and improve performance.,Initiated,Neither,12/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Automatically-Scored Writing Assessment (AWA),Federal Deposit Insurance Corporation,FDIC,Division of Administration,Mission-Enabling,,None of the above.,The Automatically-Scored Writing Assessment (AWA) is intended to significantly increase speed and reduce cost of essay scoring for the purpose of employment decisions.,"A single overall score for the essay on a scale from 1 to 5, as well as three subscale scores ranging from 1 to 5 across the following dimensions: grammar and mechanics, analysis and reasoning, as well as organization and structure.",Operation and Maintenance,"Rights-Impacting ",1/17/2018,1/17/2018,1/17/2018,,Developed with contracting resources.,FDIC does not use PIIDs,No,,,No,N/A,Yes,No,Other,FDIC's Enterprise Data Catalog is still being implemented,N/A,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",Other,"We have requested and received detailed information from the vendor supporting the validity and reliability of the AI tool. The FDIC AI Coordination Committee is currently developing an enterprise strategy for AI risk management and governance to ensure appropriate monitoring, evaluation, and mitigation of risks from the use of AI, including this use case.",Yes,No – agency does not have access to source code.,,Yes,PDRI Performance-Fit,Less than 6 months,Other,Various intake processes are available depending on the nature and scope of the request,Other,Various resource request processes are available depending on the nature and scope of the request,Other,Communication timeliness depends on the process relevant to the request. Many processes are automated and status updates are push instantaneous or are available on demand. Other manual processes involve the requestor at each step for transparency.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Structured Interview (SI) tool,Federal Deposit Insurance Corporation,FDIC,Division of Administration,Mission-Enabling,,None of the above.,"The FDIC seeks to improve the personnel selection process by automating the scheduling, creation, administration, scoring, and maintenance of structured interview materials using an online platform. ","The SI (Interview) tool will securely house a large number of internally developed, competency-based structured interview questions.",Initiated,Neither,8/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, FDIC Predictive Budget Analysis,Federal Deposit Insurance Corporation,FDIC,Division of Finance,Mission-Enabling,,Creating visual representations of data sets for reports and presentations using AI.,Take in budget data for reporting and prediction; Analyze and explain differences in budget and actuals data. Reduces time and work to produce the report; increases response time to react and review data; allows deeper dives.,Output is an explanation on gaps in data that are identified and why it was identified.,Initiated,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, CFOO Assessment Chatbot,Federal Deposit Insurance Corporation,FDIC,Division of Finance,Mission-Enabling,,None of the above.,"Compiling data from external FDIC website and targeted documents to answer public questions about Bank Assessments that are currently available on FDIC gov. Makes information more readily available to the public, searchable and readable.",An text-based answer to a question the user asks the bot.,Acquisition and/or Development,Neither,2/15/2022,4/4/2022,,,Developed in-house.,,No,,,,,,,Other,FDIC's Enterprise Data Catalog is still being implemented,"CFOO (Chief Financial Officer Organization) Assessment Chatbot use the FDIC Assessment data set for training. The data contains assessment related data elements. The assessment data set is internally/externally available for all the FDIC employees, contractors and the public through FDIC.gov external site.","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,,,,Yes,External Chatbot Rasa (ECR) - Assessment Chatbot,Less than 6 months,Other,Various intake processes are available depending on the nature and scope of the request,Other,Various resource request processes are available depending on the nature and scope of the request,Other,Communication timeliness depends on the process relevant to the request. Many processes are automated and status updates are push instantaneous or are available on demand. Other manual processes involve the requestor at each step for transparency.,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, Goals and Workload Assumptions,Federal Deposit Insurance Corporation,FDIC,Division of Finance,Mission-Enabling,,Summarizing the key points of a lengthy report using AI.,Summarize divisional efforts into a narrative taking into consideration staff numbers and competencies. Reduced time to receive answers based on data.,A visualization of effectiveness of divisional projects and a recommendation on how to fill the gap on missing elements or competencies,Initiated,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, CFOO Admin Chatbot,Federal Deposit Insurance Corporation,FDIC,Division of Finance,Mission-Enabling,,None of the above.,"Compiling data from internal FDIC website and targeted documents to answer questions about CFOO admin. Makes information more readily available to the public, searchable and readable.",An text-based answer to a question the user asks the bot.,Operation and Maintenance,Neither,9/1/2022,1/10/2023,6/13/2023,,Developed in-house.,,No,,,No,N/A,No,Yes,Other,FDIC's Enterprise Data Catalog is still being implemented,CFOO (Chief Financial Officer Organization) Admin Chatbot use the FDIC CFOO Administrative data set for training. The data contains administrative data elements. The administrative data set is internally available for all the FDIC employees and contractors.,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Internal Chatbot Rasa (ICR) - Admin Chatbot,6-12 months,Other,Various intake processes are available depending on the nature and scope of the request,Other,Various resource request processes are available depending on the nature and scope of the request,Other,Communication timeliness depends on the process relevant to the request. Many processes are automated and status updates are push instantaneous or are available on demand. Other manual processes involve the requestor at each step for transparency.,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, CFOO Travel Chatbot,Federal Deposit Insurance Corporation,FDIC,Division of Finance,Mission-Enabling,,None of the above.,"Compiling data from internal FDIC website and targeted documents to answer questions about FDIC travel. Makes information more readily available, searchable and readable.",An text-based answer to a question the user asks the bot.,Operation and Maintenance,Neither,8/10/2020,9/1/2020,12/21/2021,,Developed in-house.,,No,,,No,N/A,No,Yes,Other,FDIC's Enterprise Data Catalog is still being implemented,CFOO (Chief Financial Officer Organization) Travel Chatbot use the FDIC Travel Policy data set for training. The data contains travel policy data elements. The travel policy data set is internally available for all the FDIC employees and contractors.,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Internal Chatbot Rasa (ICR) - Travel Chatbot,6-12 months,Other,Various intake processes are available depending on the nature and scope of the request,Other,Various resource request processes are available depending on the nature and scope of the request,Other,Communication timeliness depends on the process relevant to the request. Many processes are automated and status updates are push instantaneous or are available on demand. Other manual processes involve the requestor at each step for transparency.,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, CFOO Transactional Data Analysis,Federal Deposit Insurance Corporation,FDIC,Division of Finance,Mission-Enabling,,Creating visual representations of data sets for reports and presentations using AI.,"AI assistance in monitoring financials/invoices for proper submittal, duplicate payments, and other abnormalities. Reduced time to catch errors, perform reviews, make data visualizations available.","A recommendation on when a gap is identified, why it was identified, and possible causes",Initiated,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, FDIC Staffing Model,Federal Deposit Insurance Corporation,FDIC,Division of Finance,Mission-Enabling,,Creating visual representations of data sets for reports and presentations using AI.,"Monitor retention rate, retirement planning, and knowledge retention and succession. Reduced time to receive answers, allowing folks to focus on more critical mission-driven work.",A prediction on future staffing and recommendations on how to meet varied goal points,Initiated,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, FDIC Financial Audit Report,Federal Deposit Insurance Corporation,FDIC,Division of Finance,Mission-Enabling,,Summarizing the key points of a lengthy report using AI.,"Monitor employee transactions ensure regulations and policies are being followed. Reduce time to receive answers, allowing folks to focus on more critical mission-driven work.",An identification of items that all outside policy and a recommendation on if a trend needs to be addressed,Initiated,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Risk Identification and Response,Federal Deposit Insurance Corporation,FDIC,Division of Finance,Mission-Enabling,,Creating visual representations of data sets for reports and presentations using AI.,Apply a risk model to identified risks and incorporate additional data sets to identify any correlations or possible solutions. Reduce risk to corporation by being able to more quickly and accurately analyze and report out on data.,A recommendation of any correlations among risk data and a report including possible solutions,Initiated,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, FDIC Narrative Reports Generation,Federal Deposit Insurance Corporation,FDIC,Division of Finance,Mission-Enabling,,Summarizing the key points of a lengthy report using AI.,"Analyze quarterly narrative/table data points to generate a narrative report for internal use. Reduced time to receive answers, allowing folks to focus on more critical mission-driven work.",A report or table with a summary of data points for the quarter with visualizations,Implementation and Assessment,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, FDIC Investment Portfolio Management ,Federal Deposit Insurance Corporation,FDIC,Division of Finance,Mission-Enabling,,None of the above.,"Analyze investment history and provide an additional predictive analysis for investments. Reduced time to receive answers, allowing folks to focus on more critical mission-driven work, act as an additional investment analysis review for staff to compare with.",A prediction of the outcome of possible investment choices and a recommendation of which would do best for the current portfolio,Initiated,Neither,11/11/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Analytics on Loan Data,Federal Deposit Insurance Corporation,FDIC,Division of Resolutions and Receiverships,Other,Financial Regulation,None of the above.,"This reduces the effort to analyze a portfolio of received failed bank loans and provides data supported strategies for disposition of the portfolio. This reduces the effort to analyze a portfolio of received failed bank loans which is a time- and labor-intensive process, and provides data supported strategies for dispositioning the portfolio.",Formatted data set with elements extracted from unstructured data files/documents into a readily usable data to support disposition portfolio strategies.,Initiated,Neither,4/25/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Analytics on Securities Data,Federal Deposit Insurance Corporation,FDIC,Division of Resolutions and Receiverships,Other,Financial Regulation,None of the above.,"This reduces the effort to analyze a portfolio of received failed bank loans and provides data supported strategies for disposition of the portfolio. This reduces the effort to analyze a portfolio of received failed bank loans which is a time- and labor-intensive process, and provides data supported strategies for dispositioning the portfolio.",Formatted data set with elements extracted from unstructured data files/documents into a readily usable data to support disposition portfolio strategies.,Initiated,Neither,4/25/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Business Data Extraction from Loan Files,Federal Deposit Insurance Corporation,FDIC,Division of Resolutions and Receiverships,Other,Financial Regulation,None of the above.,"Format data from unstructured documents to a structured format in a timely fashion and at scale, which will reduce cost and effort. This reduces the effort to analyze a portfolio of received failed bank loans and provides data supported strategies for dispositioning the portfolio.",Formatted data set with elements extracted from unstructured data files/documents.,Initiated,Neither,7/26/2022,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Data Extraction from Security Prospectus,Federal Deposit Insurance Corporation,FDIC,Division of Resolutions and Receiverships,Other,Financial Regulation,None of the above.,"Format data from unstructured documents to a structured format in a timely fashion and at scale, which will reduce cost and effort. This reduces the effort to analyze a portfolio of received failed bank loans and provides data supported strategies for dispositioning the portfolio.",Formatted data set with elements extracted from unstructured data files/documents.,Initiated,Neither,7/26/2022,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Deposit Insurance Determination,Federal Deposit Insurance Corporation,FDIC,Division of Resolutions and Receiverships,Other,Financial Regulation,None of the above.,"Extract and format data from unstructured documents, which will reduce the cost and effort to release or retain provisional holds placed on accounts for banks exceeding $250M . ",Structured data sets that can be analyzed by Claims Specialists.,Initiated,Neither,7/26/2022,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Initial Deposit File Analysis,Federal Deposit Insurance Corporation,FDIC,Division of Resolutions and Receiverships,Other,Financial Regulation,None of the above.,Evaluate multiple data files in different formats to extract and normalize the inputs into a rationalized data set with the key elements needed for analysis by other functional areas within FDIC.,List of accounts and assets with key data elements needed for insurance determination and asset valuation.,Initiated,Neither,7/26/2022,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Owned Real Estate (ORE) Property Appraisal,Federal Deposit Insurance Corporation,FDIC,Division of Resolutions and Receiverships,Other,Financial Regulation,None of the above.,"Validate and enrich loan data by extracting data from Loan Underwriting Documents, Title Documents, and Property Appraisal Documents, which will reduce the time to value a loan portfolio and improve the quality of the valuation.",List of property records with current and historical appraised values and historical sale transaction details.,Initiated,Neither,7/26/2022,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Trust Document Entity Extraction,Federal Deposit Insurance Corporation,FDIC,Division of Resolutions and Receiverships,Other,Financial Regulation,None of the above.,"Extract key data elements, such as grantor/settler and beneficiaries, from Trust Agreements. This will reduce the time and cost to transpose information from Trust Agreements.","The extracted data, along with the original Trust Agreement, will be presented to users through an application user interface.",Acquisition and/or Development,Neither,6/7/2024,6/10/2024,,,Developed in-house.,,No,,,,,,,Other,FDIC's Enterprise Data Catalog is still being implemented,"We used open source, sample trust agreements available on the web, as well as an internal repository of Trust Agreements retained by the FDIC after they were used in earlier insurance determination activities.","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,,,,Yes,Resolution and Receivership Management Portal,Less than 6 months,Other,Various intake processes are available depending on the nature and scope of the request,Other,Various resource request processes are available depending on the nature and scope of the request,Other,Communication timeliness depends on the process relevant to the request. Many processes are automated and status updates are push instantaneous or are available on demand. Other manual processes involve the requestor at each step for transparency.,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Trust Document Entity Extraction - LLM,Federal Deposit Insurance Corporation,FDIC,Division of Resolutions and Receiverships,Other,Financial Regulation,None of the above.,"DRR’s objective is to minimize the current processing time for monetary claims related to Trust Accounts following any bank closure. The current process involves claim specialists manually reviewing trust agreements to match and verify customer form inputs. The review process is labor intensive and may require further reviews by legal experts, hence, prolonging the processing time. This use case is to automate this process using a Gen AI powered function that can extract text from Trust Documents to match/validate customer inputs.","The extracted data, along with the original Trust Agreement, will be presented to users through an application user interface.",Initiated,Neither,6/7/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Post Examination Survey Comment Analysis,Federal Deposit Insurance Corporation,FDIC,Office of the Ombudsman,Other,Financial Regulation,Summarizing the key points of a lengthy report using AI.,"(1) Numerous comments written by different people that are challenging to group in a meaningful way. (2) Categorize the various comments to enhance analysis and understanding of the asks. ","To be determined, but potentially slotting whole or parts of comments in various categories. Potentially a table with a column of comments and rows of labels. ",Initiated,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Extracting IT information,Federal Deposit Insurance Corporation,FDIC,Division of Risk Management Supervision,Other,Financial Regulation,Searching for information using AI.,"Algorithm, Language Processing Horizontal Analysis - Risk Examination (AlphaREx) analyzes unstructured narrative content in Information Technology (IT) examination workpapers and Reports of Examination (ROE) to provide meaningful, actionable insights into information. AlphaREx capabilities contribute to the FDIC’s Modernization initiative for the Supervision Modernization Business Drivers and supports the mission of the RMS Information Technology Section.  Additionally, AlphaREx was developed to enable the automation, ingestion, analysis, and visualization of the information technology and operational unstructured and structured data gathered per the supervised financial institution.  ","Information collected across the IT Profile, FDIC Review of Examinations (ROEs), and multiple other sources (both unstructured/structured data) is a critical business need and is used by FDIC to promptly identify and address IT risks and Cyber risks for the FDIC Insured Deposit Institutions (IDIs).  In addition, this information is used to effectively identify trends, patterns and areas of improvement for the InTREx program; including training and changes to the FDIC IT Examination processes.  ",Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Transcribing Structured Interviews,Federal Deposit Insurance Corporation,FDIC,Division of Risk Management Supervision,Other,Financial Regulation,Transcribing and summarizing a recorded meeting or interview using AI.,"Risk Management Supervision (RMS) Atlanta wants to pilot the use of transcription to record structured interviews. This would allow RMS interview panels to engage with the applicant without needing to focus on taking detailed notes but would allow them to refer back to structured answers as needed. Today, panel members must transcribe every word themselves, and must split their attention between the candidate and the transcription. The use of AI is meant to supplement (not replace) human note taking during the interview.",The output will be the transcription of the audio of the interview.,Acquisition and/or Development,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, New Offsite Models,Federal Deposit Insurance Corporation,FDIC,Division of Risk Management Supervision,Other,Financial Regulation,None of the above.,This machine learning (ML) model is intended to predict 1- and 2-rated banks most likely to be downgraded to 3-rated or worse. The expected benefit is to improve bank downgrade prediction using a ML technique.,Ranking of banks based on composite rating downgrade probability.,Initiated,Neither,1/1/2018,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Document Indexing,Federal Deposit Insurance Corporation,FDIC,Division of Risk Management Supervision,Other,Indexing FDIC Documents,Digitizing text from scanned documents or smart forms for archival purposes using AI.,"The Regional Automated Document Distribution and Imaging System (RADD) is an electronic document management system that manages all supervisory related correspondence between the FDIC, financial Institutions, and other regulators. When documents are received from banks, other regulators, and FDIC staff they are manually indexed using a table-driven schema. The schema includes business line, date, source, document name, and folder. All indexing, except date, is table driven and every document must be fully indexed. Currently, RMS has roughly 24 individuals that index RADD documents. RADD has close to two million documents that are accuracy indexed. These documents have unique characteristics, such cover sheets, key words, etc. The hope is we can use AI/ML to replace, or at least supplement, the manual indexing process thus reducing labor costs and improving accuracy. ",The system would output the recommended indexing data for documents received from banks and other agencies,Initiated,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, eDiscovery Organize & Search,Federal Deposit Insurance Corporation,FDIC,Legal Division,Mission-Enabling,,Searching for information using AI.,"Electronically stored information and records have become so voluminous, it is impossible to review and search without automated assistance. The Legal Division's eDiscovery vendors use a commercial off the shelf (COTS) software to perform eDiscovery related tasks. The software deploys machine learning capabilities to receive, organize, search, and identify electronically stored information in large data sets.","The AI software identifies to the software user electronically stored information (such as documents, emails, text messages, etc. that have been uploaded into the software) likely to be relevant/responsive to user-entered searches. Human users remain responsible for verifying the AI's predictions of what documents are relevant/responsive to searches.",Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Leverage AI in the Rulemaking Process Use Case ,Federal Energy Regulatory Commission,FERC,,Mission-Enabling,,None of the above.,"The AI solution will be designed to streamline the manual review of public feedback, providing insights on sentiment and key themes for FERC’s rulemaking process. Enabling FERC analysts to complete their work more efficiently and accurately saving costs, shorten the regulatory timelines, and ensure timely implementation of policies that benefit the energy industry and the public.",Initial Comment Sentiment Analysis,,Neither,1/16/2024,1/16/2024,,,Developed with both contracting and in-house resources.,89603024F0006,No,,,No,,No,Yes,Yes,,Comments submitted into the agency via a public facing application will be utilized to train.,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",Other,No demographic variables will be used.,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Microsoft Azure Commerical Environment for prototype.,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Improve Safety Inspections Use Case ,Federal Energy Regulatory Commission,FERC,,Energy & the Environment,,None of the above.,"The AI solution is intended to enhance the efficiency and accuracy of dam and LNG pipeline structure inspections by FERCs Dam inspectors. It will analyze historical and real-time images, videos, and notes to identify potential safety defects and trends. This approach reduces the manual workload of safety inspectors and improve inspection timelines. ",Digital Images for Review,,Neither,,,,,,89603024F0006,No,,,No,,No,Yes,Other,The use case will build the repository for this use case.,Images obtained by the AI will build a repository for training.,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",Other,No demographic variables will be used.,Yes,,,No,,,Other,"This use case has not been initiated at the time of this reporting. Once initiated, the answer will be yes to the IT infrastructure provisioning.",Yes,,Other,"This use case has not been initiated at the time of this reporting. Once initiated, the answer will be yes to the IT infrastructure provisioning.",Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Enhance Market Surveillance and Fraud Detection Use Case ,Federal Energy Regulatory Commission,FERC,,Mission-Enabling,,None of the above.,"The AI solution aims to assist staff in analyzing large and complex market datasets for market integrity and compliance purposes. It will process and interpret vast amounts of trading data at high speed, identifying anomalies, trends, and potential regulatory violations. This capability will help maintain fair and transparent markets, enabling staff to focus on strategic decision-making. The AI’s efficiency and precision will save time, improve compliance, and uphold public trust in market operations.",Initial Analysis of market data,,Neither,,,,,,89603024F0006,No,,,No,,No,Yes,Yes,,We have not started work on this use case and have not identified specific data sets needed to support all market surveillance activities. ,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",Other,No demographic variables will be used.,Yes,,,No,,,Other,"This use case has not been initiated at the time of this reporting. Once initiated, the answer will be yes to the IT infrastructure provisioning.",Yes,,Other,"This use case has not been initiated at the time of this reporting. Once initiated, the answer will be yes to the IT infrastructure provisioning.",Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Support Interconnection Request Responses Use Case ,Federal Energy Regulatory Commission,FERC,,Energy & the Environment,,None of the above.,"The AI solution will be designed to address the growing backlog of project interconnection requests and reduce the average three-year wait time. By accelerating the processing of proposals, the AI will help FERC with cost savings and shorting the timelines of requests. ",Request analysis of interconnection requests,,Neither,,,,,,89603024F0006,No,,,No,,No,Yes,Yes,,We have not started work on this use case and have not identified specific data sets needed to support all interconnection requests. ,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",Other,No demographic variables will be used.,Yes,,,No,,,Other,"This use case has not been initiated at the time of this reporting. Once initiated, the answer will be yes to the IT infrastructure provisioning.",Yes,,Other,"This use case has not been initiated at the time of this reporting. Once initiated, the answer will be yes to the IT infrastructure provisioning.",Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, LexisNexis,Federal Housing Finance Agency,FHFA,OGC,Mission-Enabling,N/A,Generative AI-assisted legal research tool,Generative AI-assisted legal research tool,"LexisNexis uses generative AI to search for and summarize legal information, resulting in more efficient legal work for FHFA.",Initiated,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Neural Networks for FHFA Modeling Analytics Platform (FMAP),Federal Housing Finance Agency,FHFA,DHMG,Mission-Enabling,N/A,None of the above.,Help researchers optimize the framework and specifications of the production of the Single-Family FHFA Modeling Analytics Platform (FMAP).,"This tool uses neural networks to identify nonlinearities, anomalies, and important variables in loan-level mortgage data, which helps FHFA staff improve forecasts from the Single-Family Modeling Analytics Platform.",Acquisition and/or Development,Neither,1/8/2023,1/8/2023,,N/A,Developed in-house.,N/A,No,,,No,N/A,No,No,No,N/A,N/A,"Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",N/A,N/A,Yes,"Yes – agency has access to source code, but it is not public.",N/A,No,Machine Learning for SF FMAP specification  ,Less than 6 months,Yes,N/A,No,N/A,No,N/A,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, Phishing Email Identification,Federal Housing Finance Agency,FHFA,OCOO,Mission-Enabling,,Prioritizing and categorizing incoming emails using AI.,"Knowbe4 PhishER is a tool to report and manage suspicious emails. This tool use ML and AI to analyze suspicious emails to identify if the email is spam, phishing, or malicious. The primary benefit of this tool is to automate responses without requiring human intervention to analyze each email that end users identify as suspicious.",Output is a response to suspicious emails without requiring a cybersecurity specialist to analyze hundreds of emails. This tool reduces time to respond to end user requests concerning suspicious activity.  ,Implementation and Assessment,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, WestLaw,Federal Housing Finance Agency,FHFA,OGC,Mission-Enabling,N/A,Generative AI-assisted legal research tool,Generative AI-assisted legal research tool,"WestLaw uses generative AI to search for and summarize legal information, resulting in more efficient legal work for FHFA.",Initiated,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Public Comments Summarization and Topic Classifications Pilot Project,Federal Housing Finance Agency,FHFA,DBR,Mission-Enabling,N/A,None of the above.,To summarize and classify comments received by FHFA during public comment processes.,"This tool uses natural language processing to rapidly summarize and classify certain comments received from the public, increasing FHFA staff efficiency.",Initiated,Neither,10/1/2023,N/A,N/A,N/A,Developed in-house.,N/A,No,,Request for Comments analysis,No,N/A,No,No,Other,Data consist solely of public comments received by FHFA.,Publiccomments,"Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",N/A,N/A,Yes,"Yes – agency has access to source code, but it is not public.",N/A,No,Public Comments AnlysisTool Pilot Project (PCAT,Less than 6 months,Yes,N/A,No,N/A,No,N/A,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, Virtual Acquisition Office (VAO) Ally ,Federal Housing Finance Agency,FHFA,OBFM,Mission-Enabling,N/A,Searching for information using AI.,"VAO Ally is a built-in, useable feature as part of the overall VAO subscription. VAO Ally can reduce the amount of time necessary to research acquisition-related policy/guidance questions and scenarios by parsing through multiple information resources and providing responses/answers based on a user’s question searched.","VAO Ally provides a summarized response/answer to the user’s question within the overall web-based VAO subscription. Additionally, VAO Ally provides the source materials (e.g., VAO content, regulatory resources such as the FAR, etc.) that supported its answer/output.",Implementation and Assessment,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, "Analytics,, Search, Queries and LLM integration for data management on Oracle",Federal Housing Finance Agency,FHFA,OCIO,Mission-Enabling,N/A,Searching for information using AI.,"Oracle Enterprise 23AI has built in AI for predictive analytics, AI Vector search, Natural Language queries and integration with Large Language Models","Oracle Enterprise 23AI has built in AI for predictive analytics, AI Vector search, Natural Language queries and integration with Large Language Models",Acquisition and/or Development,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Python modules to support AI/ML,Federal Housing Finance Agency,FHFA,OCIO,Mission-Enabling,N/A,Inputting large amounts of data from paper forms into a digital system using AI.,Multiple Python modules are installed to support AI and ML,Multiple Python modules are installed to support AI and ML,Implementation and Assessment,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, R modules to support AI/ML,Federal Housing Finance Agency,FHFA,OCIO,Mission-Enabling,N/A,Inputting large amounts of data from paper forms into a digital system using AI.,Multiple R modules are installed to support AI and ML,Multiple R modules are installed to support AI and ML,Implementation and Assessment,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Security and network monitoring using Cisco Identify Services engine,Federal Housing Finance Agency,FHFA,OCIO,Mission-Enabling,N/A,Identifying unusual patterns in system logs from a single incident report using AI.,"Improves security visibility, segmentation, and threat response by analyzing network traffic and user behavior in real-time, enabling automatic detection, isolation, and resolution of potential security risks.","Cisco Identity Services Engine (ISE) uses AI for network visibility, segmentation, and threat response by analyzing network traffic and user behavior.",Implementation and Assessment,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Desktop productivity using Microsoft 365,Federal Housing Finance Agency,FHFA,OCIO,Mission-Enabling,N/A,Collaborating in real-time using AI-assisted tools in word processors.,"To help foster smart recommendations, data insights, and enhanced collaboration.","Microsoft 365 suite (Teams, SharePoint, Excel, PowerPoint, Word) uses AI for various functionalities like language translation, smart recommendations, data insights, and enhanced collaboration.",Implementation and Assessment,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Virtual Desktop using Citrix ,Federal Housing Finance Agency,FHFA,OCIO,Mission-Enabling,N/A,Identifying unusual patterns in system logs from a single incident report using AI.,"Citrix uses AI to optimize virtual desktops, manage workloads, and enhance security with behavioral analysis and threat detection.","Citrix uses AI for optimizing virtual desktop performance, managing workloads, and enhancing security through behavioral analysis and threat detection.",Implementation and Assessment,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, "Analytics, Indexing, and Anomaly Detection for Data Management on SQL Server ",Federal Housing Finance Agency,FHFA,OCIO,Mission-Enabling,N/A,Identifying unusual patterns in system logs from a single incident report using AI.,"Predictive analytics, automated indexing, and anomaly detection in data management and queries.","SQL Server incorporates AI for predictive analytics, automated indexing, and anomaly detection in data management and queries.",Implementation and Assessment,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Workflow automation and predictive analytics using ServiceNow,Federal Housing Finance Agency,FHFA,OCIO,Mission-Enabling,N/A,Logging and analyzing time spent on tasks using AI-powered time management tools.,"Help to automate workflows, predictive analytics, and providing intelligent service management solutions.","ServiceNow utilizes AI for automating workflows, predictive analytics, and providing intelligent service management solutions.",Implementation and Assessment,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Security and Compliance in file sharing and collaboration using Kiteworks,Federal Housing Finance Agency,FHFA,OCIO,Mission-Enabling,N/A,Identifying unusual patterns in system logs from a single incident report using AI.,"Enhances security and compliance in file sharing and collaboration by detecting anomalies and potential threats in real-time.""",Kiteworks employs AI to enhance security and compliance in file sharing and collaboration by detecting anomalies and potential threats.,Implementation and Assessment,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, "Network monitoring, anomaly detection through Whats Up Gold and Flowmon integration",Federal Housing Finance Agency,FHFA,OCIO,Mission-Enabling,N/A,Identifying unusual patterns in system logs from a single incident report using AI.,"Uses AI for network visibility, anomaly detection, and security through Flowmon integration","Uses AI for network visibility, anomaly detection, and security through Flowmon integration.",Implementation and Assessment,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Mathematical formula transcription using Commonlook Online,Federal Housing Finance Agency,FHFA,OCIO,Mission-Enabling,N/A,Improving the quality of written communications using AI tools.,Commonlook Online is used to translate complex mathematical formula into plain language for publicly released documents.,Commonlook Online is used to translate complex mathematical formula into plain language for publicly released documents.,Implementation and Assessment,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Mobile Worker Productivity using Apple iOS platform,Federal Housing Finance Agency,FHFA,OCIO,Mission-Enabling,N/A,Improving the quality of written communications using AI tools.,"The Apple iOS platform provides multiple applications with Natural Language Processing (NLP) capability. This is a predictive word technology where the platfom makes an assumption in real time of the word that user is going to type, and allows for a single tap selection.",The Apple iOS platform provides multiple applications with Natural Language Processing (NLP) capability. This is a predictive word technology where the platform makes an assumption in real time of the word that user is going to type and allows for a single tap selection.,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Economic Trend Modeling,Board of Governors of the Federal Reserve System,FRB,Division of Monetary Affairs,Other,Economic & Financial,None of the above.,Additional data beyond traditional statistical models,Multiple methods calculate the probability of a economic event in the next twelve months between 0 and 1 and are aggregated,Operation and Maintenance,Neither,3/1/2019,3/1/2019,9/23/2019,,Developed in-house.,,No,,,No,,No,No,Yes,,Internal- Fixed Income,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,General Support System,Less than 6 months,No,,No,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Document Organization Tool,Board of Governors of the Federal Reserve System,FRB,Division of Reserve Bank Operations and Payment Systems,Mission-Enabling,,Searching for information using AI.,To enhance the efficiency and organization of reference materials to support adversarial analysis,Labeled documents which improve searchability,Acquisition and/or Development,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, PDF Optical Character Recognition (Text),Board of Governors of the Federal Reserve System,FRB,Division of Information Technology,Other,Economic & Financial,None of the above.,The intended purpose is to extract and to decide the correct column name extracted from a report in PDF format,The list of the appropriate column names in text format,Operation and Maintenance,Neither,9/19/2021,9/22/2021,9/26/2024,,Developed in-house.,,No,,,No,,No,,No,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,General Support System,Less than 6 months,No,,No,,No,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, PDF Optical Character Recognition (Images),Board of Governors of the Federal Reserve System,FRB,Division of Information Technology,Other,Economic & Financial,Inputting large amounts of data from paper forms into a digital system using AI.,The intended purpose is to extract text from the image embedded in PDF file,The AI system's output is the image details in text format for further analysis,Acquisition and/or Development,Neither,6/11/2024,7/17/2024,,,Developed in-house.,,No,,,No,,No,No,No,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,General Support System,Less than 6 months,No,,No,,No,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Stock Market Analysis,Board of Governors of the Federal Reserve System,FRB,Division of Research and Statistics,Other,Economic & Financial,None of the above.,To understand media narratives associated with stock market events,Time series measuring association of different news topics frequencies with market volatility,Acquisition and/or Development,Neither,9/1/2022,9/1/2022,,,Developed in-house.,,No,,,No,,No,No,Yes,,External- News Feed,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,General Support System,Less than 6 months,No,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Earnings Calls Search Tool,Board of Governors of the Federal Reserve System,FRB,Division of Supervision and Regulation,Government Services (includes Benefits and Service Delivery),,Searching for information using AI.,NLP tool that summarizes earnings calls to capture key takeaways,"Interface to query various sections of public earnings reports ",Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Incident Reporting Pattern Detection,Board of Governors of the Federal Reserve System,FRB,Division of Information Technology,Mission-Enabling,,Identifying unusual patterns in system logs from a single incident report using AI.,Automatic detection of network-based security risks,Problem notifications to administrators,Implementation and Assessment,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Consumer Related Information Classification,Board of Governors of the Federal Reserve System,FRB,Division of Supervision and Regulation,Mission-Enabling,,None of the above.,Classification of consumer related information to relevant regulations,Assignment of one or more regulations to a consumer related information,Implementation and Assessment,Neither,3/16/2023,5/1/2023,10/19/2023,,Developed in-house.,,No,,,No,,No,No,Yes,,Internal- Firm Data,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,General Support System,Less than 6 months,No,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Commercial Real Estate Index,Board of Governors of the Federal Reserve System,FRB,Division of Supervision and Regulation,Mission-Enabling,,None of the above.,Identify principal component using various data points to create a market index for commercial real estate,"A principal component, which then gets used as the index",Operation and Maintenance,Neither,10/1/2023,10/1/2023,2/1/2024,,Developed in-house.,,No,,,Yes,The data science team has a quality control framework that the use case has gone through prior to being posted to the public website.,No,No,Yes,,External - Real Estate,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,General Support System,Less than 6 months,No,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, Variable Optimization,Board of Governors of the Federal Reserve System,FRB,Division of Supervision and Regulation,Mission-Enabling,,None of the above.,Iterate through alternative lag structures to identify the optimal one for forecasting call report metrics,A forecast for select call report metrics,Implementation and Assessment,Neither,1/1/2022,2/1/2022,9/28/2023,,Developed in-house.,,No,,,No,,No,No,Yes,,Internal- FFIEC Call Report,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,General Support System,Less than 6 months,No,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Credit Fragment Analysis,Board of Governors of the Federal Reserve System,FRB,Division of Research and Statistics,Other,Economic & Financial,None of the above.,Research on credit fragments,Identification of observations,Initiated,Neither,11/1/2022,,,,Developed in-house.,,No,,,No,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Cybersecurity Monitoring Tool,Board of Governors of the Federal Reserve System,FRB,Division of Supervision and Regulation,Other,Economic & Financial,Summarizing the key points of a lengthy report using AI., Monitors emerging cyber risks to the financial sector ,A risk ranking algorithm trained on a dictionary of weighted terms that imply potential financial system risk,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Proposals and Public Comments,Board of Governors of the Federal Reserve System,FRB,Office of the Secretary,Government Services (includes Benefits and Service Delivery),,None of the above.,"The Board of Governors of the Federal Reserve System (“Board”) is developing the Proposals and Public Comments (PPC) system to electronically process and manage comments from the public on regulatory rulemakings, information collections, and other proposals (collectively, “proposals”) and to post those comments to the Board’s public website. The first release of PPC is expected in mid-November 2024. The Board’s processing of comments may use artificial intelligence (AI) to provide more efficient processing of public comments (e.g., PII redaction recommendations, spam detection).","The Board’s processing of comments may use artificial intelligence (AI) to provide more efficient processing of public comments (e.g., PII redaction recommendations, sentiment analysis, text matching, entity identification, and text similarity matching) . A human verifies the system recommendations. ",Operation and Maintenance,Neither,9/1/2023,9/1/2023,11/16/2024,,Developed in-house.,,No,,,No,,Yes,Yes,No,,External- Public Comments,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Proposals and Public Comments,Less than 6 months,No,,No,,No,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Graphic design improvements,Board of Governors of the Federal Reserve System,FRB,Division of Board Members,Other,Economic & Financial,Creating visually appealing presentations using AI-driven design suggestions.,Enhances readability of presentations,Updated slide layouts and visualizations,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Bank Subsidiary Identification,Board of Governors of the Federal Reserve System,FRB,Division of Research and Statistics,Other,Economic & Financial,None of the above.,Identifying auto loans reported as subsidiaries of the banks,Identification of loans issued by finance company subsidiaries of banks,Initiated,Neither,10/1/2024,,,,Developed in-house.,,No,,,No,,No,No,No,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Manufacturer Sentiment Analysis,Board of Governors of the Federal Reserve System,FRB,Division of Research and Statistics,Other,Economic & Financial,None of the above.,To quantify survey responses related to forecasts of industrial production,Time series measuring the sentiment of manufacturing via survey respondents,Operation and Maintenance,Neither,7/1/2022,7/1/2022,7/1/2022,,Developed in-house.,,No,,,No,,Yes,Yes,Yes,,External- Manufacturing,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,General Support System,Less than 6 months,No ,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Supply Chain Estimations,Board of Governors of the Federal Reserve System,FRB,Division of Research and Statistics,Other,Economic & Financial,None of the above.,To estimate supply chain bottlenecks,Time series measuring supply chain bottleneck sentiment data,Operation and Maintenance,Neither,12/1/2022,12/1/2022,12/1/2022,,Developed in-house.,,No,,,Yes,,No,No,Yes,,Internal- Beige Book,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,General Support System,Less than 6 months,No ,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Research And Development Analysis,Board of Governors of the Federal Reserve System,FRB,Division of Research and Statistics,Other,Economic & Financial,None of the above.,To quantify AI research and development intensity in industry,Time series measuring R&D intensity within industry,Acquisition and/or Development,Neither,8/1/2024,8/1/2024,8/1/2024,,Developed in-house.,,No,,,No,,No,No,Yes,,"Internal and External- Government and Economic Data- 10K filings, FRED Macroeconomic Data","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,General Support System,Less than 6 months,No ,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Newsfeed Data Transformations,Board of Governors of the Federal Reserve System,FRB,Division of International Finance,Mission-Enabling,,Searching for information using AI.,Applying labels for country of origin and topic to articles,"Article labels for entities, topics, and sentiment",Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Newsfeed Data Processing,Board of Governors of the Federal Reserve System,FRB,Division of International Finance,Mission-Enabling,,Searching for information using AI.,To improve the processing of news articles,"The dataset contains entity extractions, topic modeling, and sentiment analysis",Acquisition and/or Development,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Body Worn Cameras Data Management System,Board of Governors of the Federal Reserve System,FRB,Office of the Inspector General,Law & Justice,,Transcribing and summarizing a recorded meeting or interview using AI.,"Audio recordings may be transcribed to text and/or redacted. Transcripts are then labeled “unverified” until an OIG designated member reviews, edits and approves the final transcript. Automatically detects and redacts screens (computer screens, digital signs), faces and license plates captured. Prior to redacting any evidence, a special agent must first approve the redactions before sharing any evidence.",Text versions of spoken word,Acquisition and/or Development,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Market Fund Portfolio,Board of Governors of the Federal Reserve System,FRB,Division of Research and Statistics,Other,Economic & Financial,None of the above.,To assist with issuer classification of money market funds,A list of likely issuers of a particular security,Operation and Maintenance,Neither,10/1/2020,10/1/2020,5/1/2021,,Developed in-house.,,No, ,,No, ,No,No,No,,External- SEC Filings,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,General Support System,Less than 6 months,No, ,No, ,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Earnings Calls Analysis,Board of Governors of the Federal Reserve System,FRB,Division of Research and Statistics,Other,Economic & Financial,Summarizing the key points of a lengthy report using AI.,"Analyzes company transcripts to identify key words, terms, and themes.",Suite of time series indicators showing evolution of word choice and topic classification,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Sentiment Analysis of Earnings Transcripts,Board of Governors of the Federal Reserve System,FRB,Division of Monetary Affairs,Other,Economic & Financial,None of the above.,Additional insight into sentiments related to topics in bank earnings calls,"The probability that text inputs are positive, negative, or neutral and classifies them as such",Operation and Maintenance,Neither,5/23/2023,6/1/2023,10/1/2024,,Developed in-house.,,No,,,No,,No,No,Yes,,,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,General Support System,Less than 6 months,No,,No,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Anomaly Detection,Board of Governors of the Federal Reserve System,FRB,Division of Supervision and Regulation,Other,Economic & Financial,None of the above.,Improve the iterative data quality process,Suggested messages regarding data quality of subsets of firm submitted data based on similar historical messages,Operation and Maintenance,Neither,8/1/2022,8/1/2022,12/1/2022,,Developed in-house.,,No,,,No,,No,No,Yes,,Internal- QIS,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,General Support System,Less than 6 months,No,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Consumer Complaints Explorer,Board of Governors of the Federal Reserve System,FRB,Division of Consumer and Community Affairs,Other,Economic & Financial,None of the above.,Improve the classification of consumer complaints into topics using topic modeling,"Gamma value, topic number, and top five terms for the topic number for each narrative",Operation and Maintenance,Neither,3/1/2018,6/1/2018,1/1/2019,,Developed in-house.,,No,,,No,,No,No,Other,The relevant data catalog is maintained by the CFPB,External- CFPB Consumer Complaints,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,General Support System,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), and took place within during use case’s development. This documentation has not been shared beyond the data science team building the tool.",,,,,,,,,,,,,,,,,, Graphic Design Improvements,Board of Governors of the Federal Reserve System,FRB,Division of Information Technology,Other,Economic & Financial,Creating visually appealing presentations using AI-driven design suggestions.,Increase working efficiency related to image processing,Images,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Regulatory Data Analysis,Board of Governors of the Federal Reserve System,FRB,Division of Supervision and Regulation,Other,Economic & Financial,None of the above.,Improved data quality for stakeholders,Values for various predicted percentile levels for a given reporter are provided to an analyst to compare to current reported values,Operation and Maintenance,Neither,6/1/2023,6/1/2023,9/9/2024,,Developed in-house.,,No,,,No,,No,No,No,,Internal- Firm Data,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,General Support System,Less than 6 months,No,,No,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Decision Tree for Deposits Data,Board of Governors of the Federal Reserve System,FRB,Division of Supervision and Regulation,Other,Economic & Financial,None of the above.,Improved process efficiency and data quality. ,Predetermined variables are calculated and then filtered to identify potential outliers in the current reporting period data,Implementation and Assessment,Neither,3/1/2023,3/1/2023,8/29/2024,,Developed in-house.,,No,,,No,,No,No,No,,Internal- H.6 Money Stock Measures,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,General Support System,Less than 6 months,No,,No,,No,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Institutional Profile Dashboard,Board of Governors of the Federal Reserve System,FRB,Division of Supervision and Regulation,Education & Workforce,,Searching for information using AI.,The output provides leadership high level information about district FR Y-9C reporters to inform discussions with business partners or stakeholders,Interactive dashboards,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Novel Activities Call Report Classification,Board of Governors of the Federal Reserve System,FRB,Division of Supervision and Regulation,Other,Economic & Financial,None of the above.,The random forest model would parse out what reported categories most significantly predict inclusion on novel banking activity lists,The model identifies call report line items that are correlated with the banks on internal supervisory lists and classifies banks based on their statistical similarity to banks engaged in novel activities,Retired,Neither,7/1/2024,7/1/2024,,9/30/2024,Developed in-house.,,No,,,No,,No,No,Yes,,Internal- FDIC Deposits,"Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,General Support System,Less than 6 months,No,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, Financial News Processing,Board of Governors of the Federal Reserve System,FRB,Division of Supervision and Regulation,Mission-Enabling,,None of the above.,Assist with financial news processing,Interactive dashboards,Implementation and Assessment,Neither,6/1/2023,8/1/2023,9/1/2024,,Developed in-house.,,No,,,No,,No,No,No,,External- News Feed,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,No – agency does not have access to source code.,,Yes,General Support System,Less than 6 months,No,,No,,No,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, Bank Exam Quality Control,Board of Governors of the Federal Reserve System,FRB,Division of Supervision and Regulation,Mission-Enabling,,None of the above.,A traditional machine learning NLP model of bank performance that utilizes information in news articles among various inputs,Model outcomes are used as a component of quality control,Acquisition and/or Development,Neither,1/1/2022,1/1/2022,,,Developed in-house.,,No,,,No,,No,No,No,,External- News Feed,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,No – agency does not have access to source code.,,Yes,General Support System,Less than 6 months,No,,No,,No,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, Document Summarization Statistics,Board of Governors of the Federal Reserve System,FRB,Division of Supervision and Regulation,Mission-Enabling,,None of the above.,Provide statistics about bank review letters based on the length of the letters and the frequency of common financial terms,Statistics on letters contained in a PDF,Operation and Maintenance,Neither,1/1/2022,1/1/2022,12/22/2024,,Developed in-house.,,No,,,No,,No,No,No,,Internal- Firm Data,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,Yes,No – agency does not have access to source code.,,Yes,General Support System,Less than 6 months,No,,No,,No,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, Writing Quality Analysis Model,Board of Governors of the Federal Reserve System,FRB,Division of Supervision and Regulation,Mission-Enabling,,None of the above.,A NLP model to help leaders better understand the writing quality and consistency of documents,Provides analysis score of consistency of writing style,Initiated,Neither,1/1/2022,1/1/2022,,,Developed in-house.,,No,,,No,,No,No,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Comment Review System,Board of Governors of the Federal Reserve System,FRB,Division of Information Technology,Government Services (includes Benefits and Service Delivery),,None of the above.,"The Comment Review System (CRS) is a system used by the Board of Governors of the Federal Reserve System (“Board”) to electronically process and manage comments from the public on regulatory rulemakings, information collections, and other proposals (collectively, “proposals”). The Board’s processing of comments may use artificial intelligence (AI) to provide more efficient processing of public comments (e.g., text matching, entity identification, and text similarity matching).​ ","To assist the analyst in reviewing each comment, CRS uses traditional machine learning natural language processing (NLP) to provide text summarization, text matching with lists of topics, entity identification, and text similarity matching. In addition, CRS identifies duplicative or near-duplicative comment letters, provides full text search (including metadata properties), and provides the optionality of providing notes or labelling comments based on various metadata attributes. All public comments are reviewed in their entirety, and summaries are used to assist with these reviews.",Operation and Maintenance,Neither,2/1/2021,2/1/2021,7/1/2021,,Developed in-house.,,No,,,No,,Yes,Yes,No,,External- Public Comments,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Comment Review System,Less than 6 months,No,,No,,No,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Trading Desk Grouping,Board of Governors of the Federal Reserve System,FRB,Division of Supervision and Regulation,Other,Economic & Financial,None of the above.,Grouping Trading Desk Descriptions,An asset class type for each desk,Implementation and Assessment,Neither,1/1/2024,1/1/2024,4/1/2024,,Developed in-house.,,No,,,No,,No,No,Yes,,External - Banking Data,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,General Support System,Less than 6 months,No,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Labor Market Analysis,Board of Governors of the Federal Reserve System,FRB,Division of Research and Statistics,Other,Economic & Financial,None of the above.,Estimates factors in the labor market,Time series measuring the quantity of layoffs,Operation and Maintenance,Neither,6/1/2022,6/1/2022,4/1/2024,,Developed in-house.,,No,,,No,,No,No,Yes,,External- SEC Filings,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,General Support System,Less than 6 months,No,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Entity Name Matching Tool,Board of Governors of the Federal Reserve System,FRB,Division of Supervision and Regulation,Other,Economic & Financial,None of the above.,Improved process efficiency,Aids analysts in comparing a reported entity name to a database of legal entity names,Acquisition and/or Development,Neither,8/1/2022,8/1/2022,11/1/2023,6/30/2024,Developed in-house.,,No,,,No,,No,No,No,,External- Corporate Structure,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,General Support System,Less than 6 months,No,,No,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Time Reporting Tool,Board of Governors of the Federal Reserve System,FRB,Division of Supervision and Regulation,Mission-Enabling,,Logging and analyzing time spent on tasks using AI-powered time management tools., Automatically classify individual work activities and produce a classification report,Categorized list of activities by time,Acquisition and/or Development,Neither,1/1/2021,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Threshold Monitoring,Board of Governors of the Federal Reserve System,FRB,Division of Supervision and Regulation,Mission-Enabling,,None of the above.,"Enhance monitoring on limits thresholds, faster collaboration",​Tools serve analysts to enhance monitoring on limits thresholds and actuals,Initiated,Neither,1/1/2022,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Supply and Demand Tool,Board of Governors of the Federal Reserve System,FRB,Division of Supervision and Regulation,Mission-Enabling,,None of the above., The tool is used to monitor and project resource and supply demand​ based on defined rule sets,Resource monitor based on rule outcomes,Initiated,Neither,1/1/2020,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Outlier Detection,Board of Governors of the Federal Reserve System,FRB,Division of Supervision and Regulation,Mission-Enabling,,None of the above.,"Traditional AI to help identify, synthesize, and deliver information provided by firms.","Identify, Synthesize, and Deliver Information from firms",Initiated,Neither,6/1/2021,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Earnings Call Topic Model,Board of Governors of the Federal Reserve System,FRB,Division of Supervision and Regulation,Mission-Enabling,,None of the above.,Model used to classify topics from earnings calls,Classify Information from Earnings Calls,Initiated,Neither,1/1/2021,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Bank Performance Monitoring,Board of Governors of the Federal Reserve System,FRB,Division of Supervision and Regulation,Mission-Enabling,,None of the above.,Model used to support off site risk analysis,"Econometric model that supplements offsite risk analysis based on historical ratings",Initiated,Neither,3/1/2022,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Network management platform,Board of Governors of the Federal Reserve System,FRB,Division of Information Technology,Mission-Enabling,,Identifying unusual patterns in system logs from a single incident report using AI.,Automatic detection of network configuration issues,Real-time notifications to administrators,Implementation and Assessment,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Bank Exam Quality Control,Board of Governors of the Federal Reserve System,FRB,Division of Supervision and Regulation,Mission-Enabling,,None of the above.,A traditional machine learning NLP model of bank ratings that utilizes information in news articles among various inputs,Model outcomes are used as a component of quality control,Acquisition and/or Development,Neither,1/1/2022,1/1/2022,,,Developed in-house.,,No,,,No,N/A,No,No,No,,External- News Feed,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,No – agency does not have access to source code.,,Yes,General Support System,Less than 6 months,No,,No,,No,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, Recommendation Engine,Board of Governors of the Federal Reserve System,FRB,Division of Supervision and Regulation,Government Services (includes Benefits and Service Delivery),,Curating news articles and updates based on user preferences using AI.,Connect users with content by providing suggestions based on the user's own view history and the view history of other users who viewed similar content,"1) Recommends similar documents to users based on the document that they are currently viewing, and 2) recommends documents to users based on their document view history",Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Risk Rating Model- Community Banks,Board of Governors of the Federal Reserve System,FRB,Division of Supervision and Regulation,Government Services (includes Benefits and Service Delivery),,None of the above.,"To improve upon the risk classification framework used to tier community banks according to risk, and to tailor examination intensity according to risk",Statistical methods are used in the variable selection process for a risk model,Operation and Maintenance,Neither,5/1/2020,5/2/2020,10/30/2023,,Developed in-house.,,No,,,No,NA,No,No,Yes,,"Internal- FFIEC Call Report, Uniform Bank Performance Report","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,General Support System,Less than 6 months,Yes,,Other,"This use case does not require intensive compute resources, but yes there is a process for requesting access to the compute environment where models are trained/estimated.",Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, Risk Rating Model,Board of Governors of the Federal Reserve System,FRB,Division of Supervision and Regulation,Government Services (includes Benefits and Service Delivery),,None of the above.,"Identify a set of factors that may be helpful in predicting the likelihood that a bank will experience an adverse outcome in the future, and to tier banks into High-, Moderate- and Low-risk tiers for review ",Statistical methods are used in the variable selection process for a risk model,Implementation and Assessment,Neither,10/1/2019,10/2/2019,,,Developed in-house.,,No,,,No,NA,No,No,Yes,,"Internal- FFIEC Call Report, Uniform Bank Performance Report","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,General Support System,Less than 6 months,Yes,,Other,"This use case does not require intensive compute resources, but yes there is a process for requesting access to the compute environment where models are trained/estimated.",Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, Automatic PSC Classification,Federal Trade Commission,FTC,BCP,Other,Consumer Response and Operations,None of the above.,"FTC receives uncategorized complaints from multiple channels and contents of comments varies widely by source. ",Product and Service code,Operation and Maintenance,Neither,7/10/1905,7/10/1905,7/11/1905,,Developed with contracting resources.,,No,,,No,,No,Yes,Yes,,Consumer Sentinal Network database,,,,,No – agency does not have access to source code.,,Yes,Sentinal Network System,,Yes,,Yes,,,,,,,,,,,,,,,,,,,,,,, Group Duplicate Complaints,Federal Trade Commission,FTC,BCP,Other,Consumer Response and Operations,None of the above.,"Consumers submit multiple complaints for the same issue using various channels, which impacts the ability of Law Enforcement to quickly search and identify leads. ",Links to duplicate complaints,Operation and Maintenance,Neither,7/10/1905,7/10/1905,7/11/1905,,Developed with contracting resources.,,No,,,No,,No,Yes,Yes,,Consumer Sentinal Network database,,,,,No – agency does not have access to source code.,,Yes,Sentinal Network System,,Yes,,Yes,,,,,,,,,,,,,,,,,,,,,,, Chatbot,Federal Trade Commission,FTC,BCP,Government Services (includes Benefits and Service Delivery),,None of the above.,Delivers unattended chat 24/7 for Identity Tehft.gov and ReportFraud.gov after business hours. Assisted chat is only available during business hours. ,Chats,Operation and Maintenance,Neither,7/10/1905,7/10/1905,7/11/1905,,Developed with contracting resources.,,No,,,No,,No,Yes,Yes,,Consumer Sentinal Network database,,,,,No – agency does not have access to source code.,,Yes,Sentinal Network System,,Yes,,Yes,,,,,,,,,,,,,,,,,,,,,,, Word Cloud,Federal Trade Commission,FTC,BCP,Other,Consumer Response and Operations,None of the above.,Uses natural language processing to visually explore complaints using word cloud and assist Law Enforcement to reduce the time required to analyze search results,word cloud,Operation and Maintenance,Neither,7/10/1905,7/10/1905,7/11/1905,,Developed with contracting resources.,,No,,,No,,No,Yes,Yes,,Consumer Sentinal Network database,,,,,No – agency does not have access to source code.,,Yes,Sentinal Network System,,Yes,,Yes,,,,,,,,,,,,,,,,,,,,,,, Graph Analytics,Federal Trade Commission,FTC,BCP,Other,Consumer Response and Operations,None of the above.,Assist Law Enforcement in identifying connections between complaints,node diagram,Operation and Maintenance,Neither,7/10/1905,7/10/1905,7/11/1905,,Developed with contracting resources.,,No,,,No,,No,Yes,Yes,,Consumer Sentinal Network database,,,,,No – agency does not have access to source code.,,Yes,Sentinal Network System,,Yes,,Yes,,,,,,,,,,,,,,,,,,,,,,, Developer productivity,Federal Trade Commission,FTC,BCP,Other,Consumer Response and Operations,None of the above.,"FTC is integrating Generative AI (Gen AI), specifically Azure OpenAI GPT-4 (AOI), into our workflow to boost development efficiency. ",Application code,Initiated,Neither,7/14/1905,7/15/1905,7/16/1905,,Developed with contracting resources.,,No,,,No,,No,Yes,Yes,,N/A,,,,,No – agency does not have access to source code.,,Yes,Sentinal Network System,,Yes,,Yes,,,,,,,,,,,,,,,,,,,,,,, AI Pilot Project to Screen and Flag for Personally Identifiable Information (PII) in Digitized Archival Records,National Archives and Records Administration,NARA,,Mission-Enabling,,Digitizing text from scanned documents or smart forms for archival purposes using AI.,"NARA is using AI to automatically find and remove sensitive personal information from its digitized historical records. This will make the process faster, more accurate, and better protect privacy while allowing for greater public access to these records. The AI will also help NARA manage its vast collection and adapt to future privacy needs.","This AI use case will produce redacted records, a prioritized list for redaction, and a tool for staff to redact unpublished records.",Acquisition and/or Development,Neither,7/15/1905,7/15/1905,TBD,N/A,Developed in-house.,N/A,No,,,Yes,N/A,Yes,Yes,Yes,,Agency is strictly using its data to test the pre-trained model for the purpose of prototype work. The data is publicly available at catalog.archives.gov.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",N/A,,Yes,"Yes – agency has access to source code, but it is not public.",N/A,No,National Archive Catalog (NAC) ,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,Planned or in-progress.,"Benchmark evaluation: Testing of the AI model has involved the use of benchmarks (either publicly available or internally created) to estimate performance in real-world settings, but has not been tested in an operational environment. ",N/A,Yes – by another appropriate agency office that was not directly involved in the system’s development,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,Online - in the terms or instructions for the service.,Not in production yet,No,N/A,Direct user testing,,,,, Freedom of Information Act (FOIA) Discovery AI Pilot,National Archives and Records Administration,NARA,,Mission-Enabling,,Searching for information using AI.,"NARA is piloting the use of AI to improve its responses to Freedom of Information Act requests. This includes AI tools for finding relevant records and automatically redacting sensitive information, which will make the process faster, more accurate, and compliant with privacy laws.",This AI use case will output relevant records matching FOIA requests and redacted versions of those records with sensitive information removed.,Acquisition and/or Development,Neither,7/16/1905,7/16/1905,TBD,N/A,Developed in-house.,N/A,No,,,Yes,N/A,Yes,Yes,Yes,,NARA will train this AI use case based on previous FOIA requests that were released to public ,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,"Yes – agency has access to source code, but it is not public.",N/A,No,TBD,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,Planned or in-progress.,"Benchmark evaluation: Testing of the AI model has involved the use of benchmarks (either publicly available or internally created) to estimate performance in real-world settings, but has not been tested in an operational environment. ",N/A,Yes – by another appropriate agency office that was not directly involved in the system’s development,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,Online - in the terms or instructions for the service.,,No,N/A,Direct user testing,,,,, Auto-fill of Descriptive Metadata for Archival Descriptions ,National Archives and Records Administration,NARA,,Mission-Enabling,,Digitizing text from scanned documents or smart forms for archival purposes using AI.,"NARA is using AI to automatically create descriptions (metadata) for its digital archives. This will save archivists time, make records easier to find, and help the public better understand the information in the National Archives Catalog.","This AI use case will create more complete and informative descriptions for records, making them easier to find in the National Archives Catalog.",Acquisition and/or Development,Neither,7/16/1905,7/16/1905,TBD,N/A,Developed in-house.,N/A,No,,,Yes,N/A,No,Yes,Yes,,"The data is publicly available at catalog.archives.gov","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,"Yes – agency has access to source code, but it is not public.",,No,Description and Authority Service (DAS),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,Planned or in-progress.,"Benchmark evaluation: Testing of the AI model has involved the use of benchmarks (either publicly available or internally created) to estimate performance in real-world settings, but has not been tested in an operational environment. ",N/A,Yes – by another appropriate agency office that was not directly involved in the system’s development,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,Online - in the terms or instructions for the service.,,No,N/A,Direct user testing,,,,, AI based Semantic Search for National Archives Catalog,National Archives and Records Administration,NARA,,Mission-Enabling,,Searching for information using AI.,"NARA is using AI to create a smarter search engine for its online catalog. This new search understands the meaning behind your search, not just the keywords, and can even find hidden connections between documents. This will make research faster, easier, and more insightful.","This AI search tool will provide better search results and show connections between records, making research easier and more insightful.",Acquisition and/or Development,Neither,7/16/1905,7/16/1905,TBD,N/A,Developed in-house.,N/A,No,,,Yes,N/A,No,Yes,Yes,,Agency is strictly using its data to test the pre-trained model for the purpose of prototype work. The data is publicly available at catalog.archives.gov.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,"Yes – agency has access to source code, but it is not public.",,No,ArchisAI,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,Yes,"Benchmark evaluation: Testing of the AI model has involved the use of benchmarks (either publicly available or internally created) to estimate performance in real-world settings, but has not been tested in an operational environment. ",N/A,Yes – by another appropriate agency office that was not directly involved in the system’s development,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,Online - in the terms or instructions for the service.,,No,N/A,Direct user testing,,,,, AI Pilot for National Declassification Center (NDC) at NARA,National Archives and Records Administration,NARA,,Mission-Enabling,,None of the above.,"NARA plans to use AI to automate the declassification of documents, making the process faster, more accurate, and transparent. This will improve efficiency, ensure consistency in applying declassification rules, and ultimately make more information available to the public.","This AI use case will output declassified documents with sensitive information redacted, ready for public release.",Planned,Neither,,,,,,,,,,,,,,Yes,,,,,,,,,,TBD,,,,,,,,,,,,,,,,,,,,,,,,,,, Develop a Custom AI Model to Power Generative AI- Capabilities,National Archives and Records Administration,NARA,,Mission-Enabling,,None of the above.,"NARA plans to build a custom AI model trained on its own data, which will improve the performance of its AI applications. This will lead to more accurate and relevant results, a better user experience, and new possibilities for using AI in managing and accessing records.",This AI use case will output an AI model that is specifically trained on NARA's data and designed to meet NARA's unique needs.,Planned,Neither,,,,,,,,,,,,,,,,,,,,,,,,TBD,,,,,,,,,,,,,,,,,,,,,,,,,,, Develop a Natural Language Based Chat Interface (like ChatGPT) to Interact With the Archival Documents,National Archives and Records Administration,NARA,,Mission-Enabling,,None of the above.,"NARA plans to develop an AI-powered chat interface, similar to ChatGPT, that will allow users to easily explore its digital archives by asking questions in everyday language. This will make historical documents more accessible and engaging, leading to more efficient research and a deeper understanding of history.",This AI will create a chat interface that lets users easily get information from NARA's archives by having a conversation with it.,Planned,Neither,,,,,,,,,,,,,,,,,,,,,,,,TBD,,,,,,,,,,,,,,,,,,,,,,,,,,, "Topic Summarizer and Entity Extraction using AI ",National Archives and Records Administration,NARA,,Mission-Enabling,,None of the above.,NARA plans to use AI to automatically create descriptions for its digital objects. This will free up archivists' time and make it easier for people to find and understand the digital materials in NARA's system.,This AI use case will output enriched metadata making NARA's digital assets more accessible and easier to manage.,Planned,Neither,,,,,,,,,,,,,,,,,,,,,,,,TBD,,,,,,,,,,,,,,,,,,,,,,,,,,, Automated Data Discovery and Classification Pilot ,National Archives and Records Administration,NARA,,Mission-Enabling,,None of the above.,"NARA plans to test how well AI can automatically organize and categorize its data. This pilot project will help NARA understand its data better, find information faster, improve efficiency, and manage risks more effectively.","This AI pilot will categorize data, improve search, and potentially train AI to identify new document types.",Planned,Neither,,,,,,,,,,,,,,,,,,,,,,,,TBD,,,,,,,,,,,,,,,,,,,,,,,,,,, Voice to text transcription,National Transportation Safety Board,NTSB,NTSB,Mission-Enabling,,None of the above.,"Create draft voice to text transcription of investigation recordings, improving speed and efficiency of initial processing for audio recordings prior to manual transcription",Draft transcription,Initiated,Neither,9/11/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Semantic Similarity Text Search,National Transportation Safety Board,NTSB,NTSB,Mission-Enabling,,Searching for information using AI.,"Semantic search of agency library of safety recommendation and report texts, improving speed and efficiency of researching previously identified transportation safety topics",Search results identifying semantically similar narrative in previously published reports and safety recommendations,Initiated,Neither,7/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Synthetic Data Generation,Pension Benefit Guaranty Corporation,PBGC,Office of Information Technology (OIT) ,Other,N/A,None of the above.,"To create artificial data that mimics the features, structures, and statistical attributes of end user/pensioner data","Unlike real data, which may contain sensitive or Personally Identifiable Information (PII), the synthetic data ensures data privacy, while enabling data analysis, research, and software testing",Initiated,Neither,11/1/2022,,,N/A ,Developed with both contracting and in-house resources.,,No,,Using website resources to assist organizations with compliance,No,N/A ,No,No,No,,N/A. Currently conducting market research to identify requirements for acquisition/implementation. ,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",N/A,,No,No – agency does not have access to source code.,n/a,No,N/a. Acquisition has not been completed as of yet. ,,No,,No,,,,,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,Planned or in-progress.,No testing: No testing of the model to simulate performance in an operational environment has been conducted thus far.,TBD. Currently doing market research to assess potential risks involved with deployment. ,Yes – by the CAIO,,,Market researched still being conducted to determine configuration. ,None of the above,,,n/a,None of the above,,,Market research is still being conducted. ,, IT Security Monitoring,Pension Benefit Guaranty Corporation,PBGC,Office of Information Technology (OIT) ,Mission-Enabling,N/A,Identifying unusual patterns in system logs from a single incident report using AI.,Analyzing large data sets of end user activity to detect anamolies and potential security events ,Enhanced detection and reporting of potential IT Security events.,Implementation and Assessment,Neither,9/1/2022,9/1/2022,9/1/2023,N/A ,Developed with both contracting and in-house resources.,,No,,"Safeguarding the confidentiality, integrity, and availability of systems and data ",No,N/A ,No,No,No,,Raw data events to clear and concise anomalies and threats for review and resolution. ,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",N/A,,No,No – agency does not have access to source code.,n/a,Yes,Embedded within PBGC's IT Infrastructure General Support System,6-12 months,Yes,,No,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,No,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.",Risks are associated with noncompliance with NIST 800-53 Rev 5 requirements. Security control assessments performed to ensure operational itent and desired outcomes. ,Planned or in-progress,No monitoring protocols have been established: Necessary infrastructure needed to perform monitoring of AI systems in production is not available and/or a plan to monitor models has not yet been established.,No - Some individual decisions or actions require direct human oversight.,,Email,,No,n/a,None of the above,,Yes,Security analysts will manually assess logged end user activities to confirm anamoly detection. ,"No – This AI use case is not subject to the opt-out requirement because the AI functionality is solely used for the prevention, detection, and investigation of fraud or cybersecurity incidents, or the conduct of a criminal investigation.", Lexis +AI,Postal Regulatory Commission,PRC,OGC,Law & Justice,,Improving the quality of written communications using AI tools.,Legal Research,Research,Operation and Maintenance,Neither,7/1/2024,7/1/2024,7/1/2024,,Developed with contracting resources.,Sole Source,Yes,,,No,NA,No,No,Other,NA,NA,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",N/A,,No,No – agency does not have access to source code.,,No,Lexis +AI,Less than 6 months,No,,No,,No,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,No,,,,,,,,,,,,,,,, "Productivity and communications including the generation of insights, automation and workflow",Presidio Trust,PT,Information Technology,Mission-Enabling,,Improving the quality of written communications using AI tools.,"Improved productivity, collaboration, information and communications","Communications, recommendations, information",Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Ability to find information through search results and other methods,Presidio Trust,PT,Information Technology,Mission-Enabling,,Searching for information using AI.,Improved finding of information,"Search results, translation, questions answered",Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, "Pattern identification, simulations, optimization and forecasting leading to alerts, notifications and reporting",Presidio Trust,PT,Information Technology,Mission-Enabling,,Identifying unusual patterns in system logs from a single incident report using AI.,"Improved awareness of the environment, better forecasting of future events","Insights, warnings, alerts, recommendations",Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, "Scheduling and managing social media posts using AI, including sentiment analysis",Presidio Trust,PT,Information Technology,Mission-Enabling,,Scheduling and managing social media posts using AI.,"Improved communications, sentiment analysis, awareness and insights","Reports on social media, recommended communications",Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Optimized navigation for traffic and other purposes,Presidio Trust,PT,Information Technology,Mission-Enabling,,Planning travel routes using AI-driven map applications.,Improved navigation and routing,Navigation and routing,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Comment Letter Review and Analysis ,Securities and Exchange Commission,SEC,Office of Information Technology (OIT),Other,Securities & Financial Markets,Summarizing the key points of a lengthy report using AI.,Enhances the review of public comment letters.,Comment letter summaries.,Implementation and Assessment,Neither,22-Jul,22-Dec,24-Jun,,Developed with both contracting and in-house resources.,50310220F0059,No,,,No,,Yes,Yes,Yes,,Comment letter data,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",Other,Not Applicable,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Comment Letter Review and Analysis System,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Forms Filed,Securities and Exchange Commission,SEC,Division of Investment Management (IM),Other,Securities & Financial Markets,None of the above.,Assists staff in evaluating completeness of filed information at scale by automating the identification of failures to file certain forms utilized by investment companies.,In planning phase - TBD,Initiated,Neither,24-Oct,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Federal Acquisition Research Solution,Securities and Exchange Commission,SEC,Office of Acquisitions (OA),Mission-Enabling,,Summarizing the key points of a lengthy report using AI.,"Aggregates and summarizes federal acquisition policy, best practices, and templates for common issues raised for the federal acquisition workforce and stakeholders. This tool will support OA operations, training, and policy formulation.","AI-generated text aggregating and summarizing federal acquisition policy, best practices, and templates for common acquisition issues.",Operation and Maintenance,Neither,24-Aug,24-Aug,24-Aug,,Developed with contracting resources.,50310224F0144,No,,,No,,No,Yes,Other,Not Applicable,Not Applicable,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",Other,Not Applicable,No,No – agency does not have access to source code.,,No,,Less than 6 months,Yes,,Other,Not Applicable,Other,Not Applicable,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Human Resources Search,Securities and Exchange Commission,SEC,Office of Human Resources (OHR),Mission-Enabling,,Searching for information using AI.,Provides results to staff inquiries. ,Search results identifying knowledge articles and services available.,Operation and Maintenance,Neither,22-Nov,22-Apr,22-Nov,,Developed with both contracting and in-house resources.,503102-22-F-0107,No,,,No,,No,Yes,Yes,,Knowledge articles related to human resource information,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",Other,Not Applicable,No,"Yes – agency has access to source code, but it is not public.",,Yes,ServiceNow HR Service Delivery ,Less than 6 months,Yes,,Other,Not Applicable,Other,Not Applicable,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, 8-K Filing Solution,Securities and Exchange Commission,SEC,Division of Corporate Finance (CF),Other,Securities & Financial Markets,Searching for information using AI.,Searches and extracts information from certain 8-K filings (HTML files and images).,"Extracts of filing data. ",Acquisition and/or Development,Neither,24-Aug,24-Sep,,,Developed in-house.,,No,,,,,,,Yes,,Not Applicable,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",Other,Not Applicable,,,,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Training Conversation Tool ,Securities and Exchange Commission,SEC,Office of Human Resources (OHR),Education & Workforce,,Transcribing and summarizing a recorded meeting or interview using AI.,Improves communication and collaboration skills of the SEC workforce,Feedback on collaboration and communication.,Implementation and Assessment,Neither,24-Nov,24-Nov,24-Nov,,Developed with contracting resources.,NNG15SC71B-50310224F0140,No,,,No,,No,Yes,Other,Not Applicable,Not Applicable,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",Other,Not Applicable,No,No – agency does not have access to source code.,,No,,Less than 6 months,Yes,,Other,Not Applicable,Other,Not Applicable,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, Mobile Phone Artificial Intelligence Features,Securities and Exchange Commission,SEC,Office of Information Technology (OIT),Mission-Enabling,,Searching for information using AI.,Tests new mobile phone AI features. ,In planning phase - TBD,Initiated,Neither,24-Nov,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Create Work-Related Videos,Securities and Exchange Commission,SEC,Office of Human Resources (OHR),Other,Securities & Financial Markets,Creating visually appealing presentations using AI-driven design suggestions.,Allows staff to make and edit work-related videos. ,"A video is created, which may be edited.",Operation and Maintenance,Neither,24-Nov,23-Jun,24-May,,Developed with contracting resources.,47QTCA21D007B/50310223F0119/P00001,No,,,No,,No,Yes,Other,Not Applicable,Not Applicable,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",Other,Not Applicable,No,No – agency does not have access to source code.,,Yes,Vyond,Less than 6 months,Yes,,Other,Not Applicable,Other,Not Applicable,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, CF Staff Review Comments ,Securities and Exchange Commission,SEC,Division of Corporate Finance (CF),Other,Securities & Financial Markets,Searching for information using AI.,Classifies staff comments and improve CF's review process.,SEC staff comment categories,Acquisition and/or Development,Neither,24-Sep,24-Sep,,,Developed in-house.,,No,,,,,,,Yes,,Filings review data,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",Other,Not Applicable,,,,Yes,DREAM - Disclosure Review Automation,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, EDGAR Filing Certification Review Solution ,Securities and Exchange Commission,SEC,Office of Information Technology (OIT),Other,Securities & Financial Markets,Inputting large amounts of data from paper forms into a digital system using AI.,Identifies potentially non-compliant 302 certifications of EDGAR filings.,Tables containing potentially non-compliant filing data.,Implementation and Assessment,Neither,23-Jan,23-Apr,23-Dec,,Developed with contracting resources.,503102-20-F-0059,No,,,No,,Yes,Yes,Yes,,EDGAR filing data,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",Other,Not Applicable,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,AI Disclosure Analytics (AIDA),Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Evaluating Cybersecurity Alerts,Securities and Exchange Commission,SEC,Office of Information Technology (OIT),Mission-Enabling,,None of the above.,Enhances identification of anomalous cybersecurity alerts for review. ,Tables containing anomalous cybersecurity alerts.,Initiated,Neither,24-May,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, EDGAR Filing Audit Report Solution ,Securities and Exchange Commission,SEC,Office of Information Technology (OIT),Other,Securities & Financial Markets,Inputting large amounts of data from paper forms into a digital system using AI.,Utilizing a tool to identify audit reports included in EDGAR filings that are potentially non-compliant with SEC Rules and Regulations.,Tables containing potentially non-compliant filing data.,Implementation and Assessment,Neither,24-Feb,24-Apr,25-Feb,,Developed with contracting resources.,503102-20-F-0059,No,,,No,,Yes,Yes,Yes,,EDGAR filing data,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",Other,Not Applicable,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,AIDA,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Public Filing Disclosure Review,Securities and Exchange Commission,SEC,Office of the Chief Data Officer (OCDO),Other,Securities & Financial Markets,None of the above.,"Summarizes public filing disclosures and identifying differences and similarities between disclosures. ",Summaries of disclosures and answers to user-selected questions.,Implementation and Assessment,Neither,24-Aug,24-Aug,24-Oct,,Developed with contracting resources.,50310224F0089,No,,,No,,No,Yes,Other,Not Applicable,Not Applicable,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",Other,Not Applicable,No,No – agency does not have access to source code.,,No,,Less than 6 months,Yes,,Other,Not Applicable,Other,Not Applicable,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, EDGAR Technical Support Public Chatbot,Securities and Exchange Commission,SEC,EDGAR Business Office (EBO),Other,Securities & Financial Markets,Searching for information using AI.,Provides public users quick answers to technical questions about accessing EDGAR. ,Responses or links to existing technical documentation. ,Initiated,Neither,24-Nov,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, EDGAR Filing Signature Review Solution ,Securities and Exchange Commission,SEC,Office of Information Technology (OIT),Other,Securities & Financial Markets,Inputting large amounts of data from paper forms into a digital system using AI.,Identifies when required signatures are potentially missing in EDGAR filings.,Tables containing potentially non-compliant filing data.,Implementation and Assessment,Neither,23-Oct,23-Dec,25-Feb,,Developed with contracting resources.,503102-20-F-0059,No,,,No,,Yes,Yes,Yes,,EDGAR filing data,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",Other,Not Applicable,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,AIDA,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, EDGAR Filing Risk Disclosure Review Solution ,Securities and Exchange Commission,SEC,Office of Information Technology (OIT),Other,Securities & Financial Markets,Inputting large amounts of data from paper forms into a digital system using AI.,Analyzes earnings call transcripts and EDGAR filing disclosure of risk factors.,Table of summarized earnings call data and EDGAR filing comparison.,Implementation and Assessment,Neither,23-Dec,23-Dec,25-Feb,,Developed with contracting resources.,503102-20-F-0059,No,,,No,,Yes,Yes,Yes,,EDGAR filing data,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",Other,Not Applicable,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,AIDA,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Testing Developer Workflows,Securities and Exchange Commission,SEC,Strategic Hub for Innocation and Financial Technology (FinHub),Mission-Enabling,,Searching for information using AI.,Tests workflows to inform guidance for future AI use case development.,In planning phase - TBD,Initiated,Neither,24-Nov,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Exploring AI Products for Enterprise Use,Securities and Exchange Commission,SEC,Office of Information Technology (OIT),Mission-Enabling,,Searching for information using AI.,Tests web-based AI features. ,In planning phase - TBD,Initiated,Neither,24-Nov,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Name Matching,Securities and Exchange Commission,SEC,Division of Examinations (EXAMS),Other,Securities & Financial Markets,None of the above.,Identifies certain similarities in names across data sources. This allows staff to more easily compare information across data sources. ,The system outputs a number which measures how closely names match.,Operation and Maintenance,Neither,18-Feb,18-Jan,18-Feb,,Developed with both contracting and in-house resources.,SECHQ16A0013,No,,,No,,Yes,Yes,Other,Use case already developed,Registrant data,"Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",Other,Not Applicable,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,NEAT,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, Parsing of Plain Language Descriptions to Machine-Readable Notations,Securities and Exchange Commission,SEC,Division of Examinations (EXAMS),Other,Securities & Financial Markets,None of the above.,Automates conversion of plain-language text into machine-readable standardized notations. This permits ease of analysis.,The system outputs machine-readable notations.,Operation and Maintenance,Neither,22-Feb,21-Oct,22-Feb,,Developed with both contracting and in-house resources.,50310220D0012,No,,,No,,No,Yes,Other,Use case already developed,Registrant data,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",Other,Not Applicable,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,NEAT,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, Identification of Potentially Manipulative Trading Activity in Registrant Data,Securities and Exchange Commission,SEC,Division of Examinations (EXAMS),Other,Securities & Financial Markets,Searching for information using AI.,Identifies potentially manipulative trading activity for further review by an examination team.,The system identifies activity of interest.,Acquisition and/or Development,Neither,22-Mar,21-May,,,Developed with both contracting and in-house resources.,50310220D0012,No,,,,,,,Yes,,Registrant data,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",Other,Not Applicable,,,,Yes,HALSI,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Testing of Various AI Algorithms and Models ,Securities and Exchange Commission,SEC,Division of Examinations (EXAMS),Other,Securities & Financial Markets,Summarizing the key points of a lengthy report using AI.,"Experimentation using technology to determine if appropriate to enhance examiner efficiency, productivity, or effectiveness.",In planning phase - TBD,Initiated,Neither,24-Apr,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Using Natural Language Processing Techniques to Search and Analyze Certain Filings ,Securities and Exchange Commission,SEC,Division of Examinations (EXAMS),Other,Securities & Financial Markets,Searching for information using AI.,Helps search and analyze a corpus of unstructured filings. Staff may consider the results of these efforts in their examination efforts.,This system allows for search and analysis of certain filings.,Initiated,Neither,20-Apr,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Using Machine Learning/Artificial Intelligence Techniques to Predict Entities With Certain Risk Characteristics,Securities and Exchange Commission,SEC,Division of Examinations (EXAMS),Other,Securities & Financial Markets,None of the above.,Analyze data to predict entities with certain risk characteristics. Staff may consider the results of these efforts in their examination efforts. ,This system provides predictive information. ,Operation and Maintenance,Neither,14-Jan,14-Jan,15-Apr,,Developed with both contracting and in-house resources.,SECHQ113A0004,No,,,No,,No,Yes,Yes,,Filings and examination data,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",Other,Not Applicable,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,EDW,6-12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Developed Parsing Logic for Filings ,Securities and Exchange Commission,SEC,Division of Examinations (EXAMS),Other,Securities & Financial Markets,None of the above.,Help staff more efficiently search and analyze disclosure text contained in various filings.,These efforts will help to search and analyze certain filings.,Operation and Maintenance,Neither,20-Jan,23-May,24-Jul,,Developed with both contracting and in-house resources.,50310219D0008,No,,,No,,No,Yes,Yes,,Filings,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",Other,Not Applicable,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,AI Filing Parser,More than 12 months,Yes,,Yes,,No,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, Identification of Potentially Manipulative Activity in Certain Accounts,Securities and Exchange Commission,SEC,Division of Examinations (EXAMS),Other,Securities & Financial Markets,Searching for information using AI.,Identifies accounts with certain activity of interest for further review by an examination team.,The system identifies activity of interest.,Implementation and Assessment,Neither,21-Oct,20-Oct,24-May,,Developed with both contracting and in-house resources.,50310220D0012,No,,,No,,No,Yes,Yes,,Registrant and market data,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",Other,Not Applicable,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,NEAT,6-12 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Using AI to Help with Coding-Related Tasks/Questions,Securities and Exchange Commission,SEC,Division of Enforcement (ENF); Division of Economic and Risk Analysis (DERA),Education & Workforce,,None of the above.,"Helps staff with coding-related tasks, for a subset of supported languages. Staff can also use this as an educational tool to learn how to code more efficiently.","It outputs code responsive to the user’s request. It is textual, non-self-executing, and typically has non-code text explaining the logic behind the code and how to use the code.",Operation and Maintenance,Neither,24-Jan,24-Jan,24-Jun,,Developed in-house.,,No,,,No,,No,Yes,Yes,,Not Applicable,"Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",Other,Not Applicable,Yes,No – agency does not have access to source code.,,Yes,SEARCH,Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, Optical Character Recognition (OCR) for Records Analysis,Securities and Exchange Commission,SEC,Division of Enforcement (ENF),Other,Securities & Financial Markets,Digitizing text from scanned documents or smart forms for archival purposes using AI.,Assists ENF staff in extracting relevant information from digital image files. This will accelerate associated ENF investigations.,A text string of all text recognized from the associated image. ,Operation and Maintenance,Neither,24-Mar,24-Mar,24-Oct,,Developed with both contracting and in-house resources.,50310220D0012_50310221F0160 ,No,,,No,,Yes,Yes,Yes,,Image data,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",Other,Not Applicable,No,No – agency does not have access to source code.,,Yes,SEARCH,Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Microsoft Copilot for Internal Employees ,United States Agency for Global Media,USAGM,OCIO,Mission-Enabling,,Improving the quality of written communications using AI tools.,"USAGM is exploring the potential of Microsoft Copilot to enhance workplace productivity and collaboration across its internal teams. The tool leverages generative AI capabilities integrated within the Microsoft 365 suite, enabling employees to streamline workflows and focus on high-value tasks. Key features include drafting documents and presentations, summarizing content, automating reminders, managing editorial calendars, scheduling meetings, recording and transcribing conversations, and conducting natural language search for quick access to files and records. This initiative supports operational efficiency by integrating AI capabilities across collaboration, task management, and content creation workflows. ",,Initiated,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Automated Transcription Generation ,United States Agency for Global Media,USAGM,OCIO,Mission-Enabling,,Transcribing and summarizing a recorded meeting or interview using AI.,"Generative AI is being leveraged in USAGM’s media workflows to enhance efficiency and accessibility. This use case integrates Latakoo, which utilizes OpenAI's Whisper technology for transcription. With high accuracy, the system transcribes content as it is processed and passes it into USAGM’s media asset management (MAM) system, ensuring that the content is easily searchable and accessible for future use. ",,Initiated,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Content Enhancement for Articles ,United States Agency for Global Media,USAGM,OCIO,Mission-Enabling,,None of the above.,"This project uses AI integrated with our content management system (CMS) and digital platforms to assist journalists in optimizing article quality. AI generates multiple versions of titles, automates the creation of SEO-friendly tags and text, and provides concise summaries for articles. These summaries enhance user engagement by increasing the time spent on pages, particularly when readers arrive via search engines or social media. Journalists can review and adjust AI suggestions to ensure alignment with their goals. ",,Initiated,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Automated Translations ,United States Agency for Global Media,USAGM,OCIO,Mission-Enabling,,None of the above.,"This initiative uses AI integrated with our content management system (CMS) to streamline multilingual content production across USAGM’s networks. Unlike typical translation tasks, USAGM faces the unique challenge of efficiently translating content into 50-plus languages, spanning diverse linguistic and cultural contexts. The AI translation tool provides draft translations for cloned articles, enabling seamless content reuse between language teams while maintaining accuracy and cultural sensitivity. This solution is critical for ensuring high-quality, localized content delivery to audiences worldwide. ",,Initiated,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, AI-Powered CMS Guide Agent ,United States Agency for Global Media,USAGM,OCIO,Mission-Enabling,,Searching for information using AI.,"An AI-driven agent is being developed to enhance user support within the CMS. The agent will respond to natural language questions, offering tailored guidance and solutions from the CMS’s internal help documentation. ",,Planned,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Content Summarization and Audience Engagement ,United States Agency for Global Media,USAGM,OCIO,Mission-Enabling,,Searching for information using AI.,"AI tools integrated with our content management system (CMS) are being piloted to create multi-article summaries, ""smart search"" responses, and interactive highlights tailored for mobile-first users. These tools enhance content discoverability within the CMS and enable the creation of widgets like ""all you need to know"" summaries, improving user engagement and accessibility. Additionally, AI-generated interactive formats provide better comprehension for long-form articles, ensuring a seamless user experience across digital platforms. ",,Planned,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Automated Text Narration ,United States Agency for Global Media,USAGM,OCIO,Mission-Enabling,,None of the above.,AI-generated audio narrations are being explored as a tool to enhance accessibility and engagement for long-form articles. An integrated audio player would allow users to consume content in a more convenient and flexible manner. ,,Planned,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Automated Metadata Generation ,United States Agency for Global Media,USAGM,OCIO,Mission-Enabling,,None of the above.,"AI is being tested to automate the generation of descriptive metadata, such as content summaries and SEO tags. This capability aims to simplify cataloging processes, improve content discoverability, and reduce the manual effort required for metadata creation",,Planned,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Microsoft Copilot Integration,United States Commission on Civil Rights,USCCR,ASCD/IT,Mission-Enabling,,Summarizing the key points of a lengthy report using AI.,Efficiency and improved administrative tasks.,Text-based summaries and AI generated text. ,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,No – Agency did not request an extension for this use case.,No,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.",AI is not safety or rights-impacting.,AI is not safety or rights-impacting.,AI is not safety or rights-impacting.,AI is not safety or rights-impacting.,,AI is not safety or rights-impacting.,,No,AI is not safety or rights-impacting.,None of the above,,No – it is not operationally practical to offer this.,AI is not safety or rights-impacting.,Other,"AI is not safety or rights-impacting, is not required for the completion of any tasks, and is optional for individual employees to use." Relativity One ,United States Commission on Civil Rights,USCCR,OGC,Other,FOIA,Summarizing the key points of a lengthy report using AI.,Efficiency and improved administrative tasks.,Text-based summaries and AI generated text. ,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.",,,,,,,,,,,,,,, Lookout for mobile security,United States Trade and Development Agency,USTDA,N/A,Mission-Enabling,N/A,None of the above.,"Lookout uses AI and machine learning to detect and respond to cyberattacks in real time. Lookout's threat intelligence services include data forensics, incident response, and research.",A Dashboard used for monitoring activity and reports,Operation and Maintenance,Neither,12/1/2022,5/1/2023,7/1/2023,,Developed with contracting resources.,Shared Service through DHS CISA's CDM Program,No,,,No,N/A,No,No,No,,"Not applicable. The Lookout software is a commercial-off-the-shelf platform, which is provided to USTDA by the Continuous Diagnostic and Mitigation Program under the Department of Homeland Security, Cybersecurity and Infrastructure Security Agency. The platform is one of several DHS/CISA services contracted by USTDA for security.","Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",N/A,,No,No – agency does not have access to source code.,,Yes,Lookout (FEDRAMP) PK ID F1603297883,Less than 6 months,Other,Shared Service DD Form 1149,No,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Qualys Cloud Detection and Response,United States Trade and Development Agency,USTDA,N/A,Mission-Enabling,N/A,None of the above.,"Qualys uses AI to detect threats in real time, such as malware, ransomware, and unauthorized access.",A Dashboard used for monitoring activity and reports; real-time alerts when issues are discovered.,Operation and Maintenance,Neither,7/1/2021,11/1/2021,12/1/2021,,Developed with contracting resources.,Shared Service through DHS CISA's CDM Program,No,,,No,N/A,No,No,No,,"Not applicable. The Qualys tool is a commercial-off-the-shelf platform, which is provided to USTDA by the Continuous Diagnostic and Mitigation Program under the Department of Homeland Security, Cybersecurity and Infrastructure Security Agency. The platform is one of several DHS/CISA services contracted by USTDA for security.","Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",N/A,,No,No – agency does not have access to source code.,,Yes,Qualys (FEDRAMP) PK ID F1508207205,Less than 6 months,Other,Shared Service DD Form 1149,No,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Crowdstrike Falcon Threat Detection and Response,United States Trade and Development Agency,USTDA,N/A,Mission-Enabling,N/A,None of the above.,"Crowdstrike uses AI to detect and respond to threats in real-time, such as malware, ransomware, and unauthorized access.",A Dashboard used for monitoring activity and reports; real-time alerts when issues are discovered.,Operation and Maintenance,Neither,7/1/2021,11/1/2021,12/1/2021,,Developed with contracting resources.,Shared Service through DHS CISA's CDM Program,No,,,No,N/A,No,No,No,,"Not applicable. The Crowdstrike tool is a commercial-off-the-shelf platform, which is provided to USTDA by the Continuous Diagnostic and Mitigation Program under the Department of Homeland Security, Cybersecurity and Infrastructure Security Agency. The platform is one of several DHS/CISA services contracted by USTDA for security.","Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",N/A,,No,No – agency does not have access to source code.,,Yes,Crowdstrike (FEDRAMP) PK ID FR1807853629,Less than 6 months,Other,Shared Service DD Form 1149,No,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Exiger Due Diligence IQ (“DDIQ”) Research Software,United States Trade and Development Agency,USTDA,N/A,Mission-Enabling,N/A,Searching for information using AI.,"The AI utilized by Exiger’s team of analysts improves the efficiency of their research efforts into individuals and companies under consideration for partnership with USTDA by reviewing and consolidating relevant publicly available information, such as news reports.","Exiger’s DDIQ platform produces consolidated publicly available data on searched subjects for review, refinement, and analysis by Exiger’s due diligence and research professionals.",Operation and Maintenance,Neither,2/24/2024,3/1/2024,5/1/2024,,Developed with contracting resources.,1131PL24CCP91094 ,No,,,No,N/A,No,No,No,,"Not applicable. The DDIQ software is a proprietary product developed by Exiger as a commercial-off-the-shelf platform, which is available to purchase by anyone in the general public and has not been modified for government use. The DDIQ platform is one of several Exiger services contracted by USTDA for due diligence.","Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",N/A,,No,No – agency does not have access to source code.,,Yes,Exiger Insight 3PM (FEDRAMP) PK ID FR2122140784,Less than 6 months,No,,No,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Evolv WDS - OSSO,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Law & Justice,,Creating visual representations of data sets for reports and presentations using AI.,Using heat map technology to identify heat signature of known weapons.,Areas of interest for a heat map that include objects that would produce heat indicators aligned with heat signatures of known types of different weapons.,Operation and Maintenance,Both,10/1/2022,11/1/2022,2/1/2023,,Developed with contracting resources.,36C25023P0332 ,No,,,No,,No,No,No,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,N/A,Other,N/A,Other,N/A,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, VA Chat Copilot Meta Pilot,Department of Veterans Affairs,VA,OIT: Office of Information & Technology,Mission-Enabling (internal agency support),,Improving the quality of written communications using AI tools.,Administrative efficiency for VA employees ,"Chat Copilot is a generative AI interface that allows users to upload documents and query the tool about those documents. Outputs can include draft emails, draft communications, document summaries, draft project templates, responses to questions about content of documents, etc.",Acquisition and/or Development,Neither,12/1/2023,1/31/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Avigilon Camera/Search Function - OSSO,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Law & Justice,,Searching for information using AI.,A search function in a security video surveillance system that search's for points of interest in the video.,Video or picture of requested searches from the video surveillance systems.,Operation and Maintenance,Both,3/1/2022,8/1/2022,10/1/2022,,Developed with contracting resources.,,No,,,No,,Yes,No,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,N/A,Other,N/A,Other,N/A,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, CareCentra Next Level Personalized AI Health Coach,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,"This is an evidence based, randomized control trialed AI solution that focuses on precision nudging. It is intended to improve quality of care and reduce view alerts. Its focus is on identifying personalized behavioral change algorithms for each veteran to identify timing, channel, ",,Implementation and Assessment,Neither,1/1/2024,1/1/2024,5/1/2024,,Developed with both contracting and in-house resources.,CRADA: Cooperative research and development agreement,Yes,Veterans Health Administration,,No,,Yes,Yes,Other,not applicable,not applicable,"Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.","[""Race/Ethnicity"",""Sex/Gender"",""Age"",""Socioeconomic Status"",""Residency Status""]",,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Arches,Less than 6 months,Yes,,Other,not applicable,Other,n/a,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, SafePointe WDS - OSSO,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Law & Justice,,Creating visual representations of data sets for reports and presentations using AI.,Heat map detection for weapon detection software.,Areas of interest for a heat map that include objects that would produce heat indicators aligned with heat signatures of known types of different weapons.,Retired,Both,6/23/2023,6/27/2022,2/1/2024,5/1/2024,Developed with contracting resources.,,No,,,No,,No,No,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,N/A,Other,N/A,Other,N/A,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, GE Xeleris ,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,Creating visual representations of data sets for reports and presentations using AI.,Xeleris V leverages AI to accurately segment organs for quantitation and dosimetry calculations. Q.Thera AI demonstrated an average reduction of 58% in the time required for the user to process and calculate dose3.,Provide patient dosimetry calculations.,Operation and Maintenance,Both,,,,,,,,,,,,,,,,,,,,,,,Yes,MD-LITE ID: 2278,,,,,,,,,,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Genesys Cloud Contact Center,Department of Veterans Affairs,VA,OIT: Office of Information & Technology,Mission-Enabling (internal agency support),,Transcribing and summarizing a recorded meeting or interview using AI.,"Assist staff with knowledge based recommendations, conversation summarization and quality assurance metrics automation. This will allow the staff to react quicker to customer needs and spend less time researching for information or performing repetitive tasks and instead focus more on customer interactions.",Most of genesys outputs are surfacing knowledge to assist the staff with customers interactions or automating repetitive tasks such as scoring quality forms for those interactions or routing of customers upon discovered intents.,Acquisition and/or Development,Neither,2/22/2024,7/26/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Verkada Camera - OSSO,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Law & Justice,,Searching for information using AI.,Video search function inside a physical security camera solution.,Video and images from the AI searches inside the physical security camera solutions.,Acquisition and/or Development,Both,5/1/2024,9/1/2024,,,Developed with contracting resources.,,No,,,No,,Yes,No,Yes,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,N/A,Other,N/A,Other,N/A,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, MRI - GE Signa Artist,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,Creating visual representations of data sets for reports and presentations using AI.,"AIR™ Recon DL is a deep-learning-based reconstruction algorithm that enables radiologists to achieve pin-sharp images quicker. By removing noise and ringing from raw images, scans are consistently crisp. Improved signal-to-noise ratio (SNR) means scan times are cut by up to 50%, smoothing workflow and enhancing the patient experience",MRI image reconstruction.,Operation and Maintenance,Both,,,,,,,,,,,,,,,,,,,,,,,Yes,MD-LITE ID: 2278,,,,,,,,,,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Google Cloud Platform - CCAI / Dialogflow ,Department of Veterans Affairs,VA,OIT: Office of Information & Technology,Government Services (includes Benefits and Service Delivery),,None of the above.,to allow for better self service options to veterans to include virtual voice/chat agents,the output of GCP voice/chat is text to speech (TTS) and speech to text (STT),Acquisition and/or Development,Neither,7/1/2022,12/1/2022,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Axon Body Camera and DMS - OSSO,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Law & Justice,,Searching for information using AI.,"To be able to search the data repository with better results, better redaction abilities for certain capabilities and on the Fleet System only to have better license plate recognition. ","To have better outputs in searching the systems data repository for case files, evidence locker, and DMS; to have a better result when doing the redaction of evidence, files, videos and photos; and on the Fleet system only to have better results in identifying licenses plates in the licenses plate recognition part of the system.",Operation and Maintenance,Both,9/1/2020,9/26/2022,1/3/2023,,Developed with contracting resources.,,No,,,No,,Yes,No,Yes,,N/A,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,No,No – agency does not have access to source code.,,Yes,Axon,Less than 6 months,Other,N/A,Other,N/A,Other,N/A,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Ultrasound - GE Logic E10,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,Creating visual representations of data sets for reports and presentations using AI.,"Auto Preset Assistant: Automatically activates the correct preset for the anatomy being scanned Auto Abdominal Color Assistant: Detects which abdominal organ is being scanned, automatically switching to the optimal color flow parameters Auto Renal Measure Assistant: Automatically detects the kidney and measures the length, height, and width",Clinician decision support and workflow error reduction.,Operation and Maintenance,Both,,,,,,,,,,,,,,,,,,,,,,,Yes,MD-LITE ID: 2278,,,,,,,,,,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Smart AI Bot Assistant,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,"A machine learning and natural language processing algorithm will be trained to provide real-time prompts of resources to improve the quality of crisis calls for the Veterans Crisis Line (VCL). Curated, vetted resources will be prompted by keywords for responders to consider during a crisis call. Natural language processing will be used to listen and document during and after the call, organizing the documentation to include topic areas and key points. This will decrease the time spent on documentation, improve the responder workflow, and provide needed downtime for responders to debrief with colleagues and take breaks. Additionally, the chat bot could detect the need for supervisory support during a call, improving real-time communication between a VCL responder and their supervisor. The use cases for the AI will be reviewed by VA experts and subject matter experts, with the draft use cases requiring technical steps to build, pilot test, and refine.","Smart Responder Chat Bot Use Case #1 – Real time resources for VCL responders Using machine learning and natural language processing an algorithm will be trained based on feedback from the Veterans Crisis Line staff and crisis line responders to provide real-time prompts of resources to improve the quality of the crisis call. Curated, vetted resources will be prompted by key words for the responder to consider using during a crisis call. Smart Responder Chat Bot Use Case #2 – AI documentation for VCL responders To improve the time each Veterans Crisis Line responder spends on documentation during and after a crisis call, natural language processing will be used to listen and document during and after the call. The documentation will be organized to include the topic areas and key points required in documentation and provide a final draft editable by the responder. This use case will; decrease the time spent by responders on documentation, improve the responder workflow for VCL operations, and provide needed downtime away from both answering calls and documentation for responder to debrief with colleagues, accomplish other tasks and take necessary breaks to improve overall responder wellbeing. Smart Responder Chat Bot Use Case #3 – AI for supervisory support during a call Using machine learning and natural language processing including sentiment analysis, a chat bot could detect the need for a responder to seek supervisory support. Improving the real-time communication between a VCL responder and their supervisor during a call that is challenging (eg. Inappropriate language, escalating rapidly, etc.) could be a useful use case.",Initiated,"Rights-Impacting ",9/19/2024,,,,Developed with both contracting and in-house resources.,,Yes,Veterans Health Administration,,No,,No,No,Other,TBD,TBD,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,No – agency does not have access to source code.,,No,,6-12 months,Other,TBD,Other,TBD,Other,TBD,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Radiology - Siemens YSIO Max,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,Creating visual representations of data sets for reports and presentations using AI.,Bone supression: Supressess bones on digital images without the need for two exposures. Diamond View: improves image contrast and detail to improve image quality.,Increase radiology digital image quality.,Operation and Maintenance,Both,,,,,,,,,,,,,,,,,,,,,,,Yes,MD-LITE ID: 2278,,,,,,,,,,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Enterprise Precision Scanning and Indexing (EPSI) Next: Enhancing Healthcare Services for Veterans,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,Summarizing the key points of a lengthy report using AI.,Generative AI required for summarization and note creation.,The output generates a summary of the patient health record for the EPSI product indexer workflow.,Acquisition and/or Development,Neither,6/2/2021,8/2/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Nursing Proficiency Coach & Nurse Proficiency Evaluator...and beyond!,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Education & Workforce,,None of the above.,"Two custom GPTs have been created: Nurse Proficiency Writer and Nursing Proficiency Coach. A GS version is in development. The Nurse GPTs are used to compare the nurse's self-input against 600+ functional statements and standards (externally accessible or approved by the Office of Nursing Service). It evaluates the nurse's performance against the standards, reports on positive findings, identifies weak areas, provides an estimated rating, and grade assessment. The interactive GPT allows nurses to have a dialog to better understand how their practice can be improved. Many of our 90,000+ RNs are not proficient in understanding the standards nor adept in writing performance reports. This leads to end-user frustration, quality issues with content, submission gaps compared to the standard, interrater reliability variances, and a host of other challenges with the our current process. The tool aids the nurses in developing the content for their ePerformance submission. No PHI is allowed in nursing proficiencies (by standard), and PHII is scrubbed prior to submission by the nurse. "," The output takes the nurse's input and returns a summary of each practice dimension and positive accomplishments offering suggestions where the nurse can improve.  The tool can help the nurse in generating a written summary for ePerformance submission.  This is a tremendous time saver and improves the quality of their submission.  Most importantly, the Custom GPT does not impact the staff rating.  It is a tool is designed to help the nurse.  The nurse is ultimately responsible for the content and final submission (data accuracy and validity), and the proficiency evaluations are the responsibility of the Supervisor with knowledge regarding the staffs performance.  The improvment in content can greatly assist the Supervisor in their evaluation process (win-win).       ",Implementation and Assessment,Neither,8/20/2024,8/27/2024,10/5/2024,,Developed in-house.,,No,,,No,,No,No,No,," Nursing Functional Statements ","Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,No – agency does not have access to source code.,,No,"Custom ChatGPT paid for by owner, external to VA. ",Less than 6 months,No,,No,,Other," N/A, externally developed Custom ChatGPT paid for by the employee for nurse use. ",None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Parable 3D Wound Care Management System,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,Creating visual representations of data sets for reports and presentations using AI.,"Traditional wound care methods call for hand measurements to be taken of the patients wound, increasing imprecision that comes with manual measurements, along with the increased contact with the wound and potential for infection. Parable 3D Wound Care Management System aims to automate the process of wound care by using traditional computer vision based volumetric measurements to capture images of the wound and create 3D polygon mesh representation of said wound. The main benefit of the system is the automated, contactless measurements that will reduce vectors for infection and alleviate the difficulty, imprecision, and variability of manual measurements. This allows the clinicians to spend more time on patient care, analyzing wounds, and encourage proper healing for the patient.","2D and 3D volumetric measurements are computed based on the segmented 3D reconstruction to produce length, width, surface area, depth, and volume measurements.",Operation and Maintenance,Both,,,,,,,,,,,,,,,,,,,,,,,Yes,Parable ID: 665,,,,,,,,,,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, VEO Virtual Analyst Proof of Concept,Department of Veterans Affairs,VA,OIT: Office of Information & Technology,Government Services (includes Benefits and Service Delivery),,Searching for information using AI.,The AIDV Dev/Test environment will provide a proof of concept to provide generative AI capabilities in CX Insights and SDP. This will show capability to allow users to intuitively and easily work with dummy data and provide a sandbox to develop generative AI tools. The development of Virtual Analyst tool will provide value to the veteran by using generative AI to predict and improve services to the veteran.,Outputs will include data-informed English-language responses to English-language questions asked about VEO machine learning models and customer experience metrics. These outputs will be informed by dummy versions of these data.,Acquisition and/or Development,Neither,2/20/2024,4/25/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, GE Portable Critical Care Suite 2.x,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf18/K183182.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Patient Care Systems Integration Program (PCSIP),Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,Transcribing and summarizing a recorded meeting or interview using AI.,"The PCSIP AI is intended to improve the access to care for Veterans by increasing capacity of VAMCs up to 20% without adding more staff or resources. The expected benefits of the AI are listed below. (1) Between 20% to 50% reduction in charting time. (2) Reduced cancellations/delays/no-shows between 5% and 40% depending on type of appointment of patient. (3) Between 5% and 20% increase in number of scheduled cases in surgery, primary care, radiology, behavioral health, and specialty care. (4) Between 5% and 20% improvement in Door-to-Doc times and Length of Stay in Emergency Department. (5) Between 5% and 20% improvement in patient flow between ED and Wards; and between Surgery and Wards. ","All outputs listed below are written to patient record in Vista/CPRS and Cerner after review by care providers and electronic signature by them. 1. Ambient Dictation and AI generates, a. ED Triage Note, b. ED Evaluation Note, c. Columbia SSRS, d. After Visit Summary, e. Follow up appointment note, f. Patient Letter g. Care Pathway Mapping for generating list of diagnostic orders, medication orders etc. 2. Radiology a. Resulting Note in MRI, CT, Ultrasound, X-Ray, and others. b. Protocoling by Radiologist and Technologist notes written to Vista 3. Surgery a. Scheduling optimization b. Preop readiness checks and reports c. Preop Instruction Compliance d. Discharge Instruction Compliance e. ERAS protocol compliance 4. Behavioral Health a. Generation of Columbia-SSRS and Comprehensive SSRS for suicide reduction b. Video monitoring of mental health inpatients 5. Primary Care a. Ambient Dictation driven note generation. b. Outside records consolidation for Patient Summary extracted from faxes. 6. Instant availability of all the relevant patient information at the point of care based on schedule of the patient and location of the patient and care provider at the point of care.",Implementation and Assessment,Both,,,,,,,,,,,,,,,,,,,,,,,Yes,PCSIP ID: 1258,,,,,,,,,,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Billing Claims Prediction,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,The intended purpose of the AI is to support Revenue Operations Billing activities by providing a propensity to bill flag based on previous user activity in the Revenue Operations Workflow Tool. The output of the AI is strictly intended to be used for decision support for users who are determining whether an encounter is billable or not. ,"The model was trained on Third Party Community Care Billing and Facility Revenue Activity Codes 10, 30, 31, 32, 33, 34, 35 applied from 2021-2023. Initial run at Florida Caribbean Consolidated Patient Account Center (FCCPAC) was trained on 1.2 Million accounts. By training the model on previous activity data, an account can be classified as likely to have a billable or non-billable outcome.",Implementation and Assessment,Neither,2/1/2024,2/1/2024,6/14/2024,,Developed with contracting resources.,VA118-16-D-1006 36C10B22N10060040,No,,,No,N/A.,Yes,Yes,Yes,,"1.2 million claims trained from Community Care Claims Data from Community Care Reimbursement System (CCRS), Electronic Claims Adjudication Management System (eCAMS) and VACS sourced via Central Data Warehouse (CDW).","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,N/A,Yes,"Yes – agency has access to source code, but it is not public.",N/A,Yes,Revenue Operations Workflow Tool - RPA,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, ScienceLogic Artificial Intelligence Operations (AIOPS) Software Subscriptions with Maintenance and Professional Support Services.,Department of Veterans Affairs,VA,OIT: Office of Information & Technology,Other,Cloud,None of the above.,Performance monitor of cloud applications ,"The SL1 AIOps solution enables VA to diagnose reasons for outages and rapid capability to respond, resolve and prevent such outages across the full domain.",Operation and Maintenance,Neither,9/24/2020,9/24/2020,9/24/2020,,Developed with both contracting and in-house resources.,NNG15SC91B/36C10B20F0417,No,,,No,N/a -,No,Yes,No,,N/a,"Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.","[""Other""]",None - Monitor applications in the cloud,No,No – agency does not have access to source code.,,Yes,VA Enterprise Cloud ,Less than 6 months,Other,For the VA's cloud,Other,For the cloud,Other,Cloud,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, ECG/EKG Machines- Interpretation of Results,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,This software aids with interpretation of clinical Electrocardiogram tracings for VA clinicians. ,"The system integrates w/ CPRS and outputs patient demographics on the ECG print-out. It then prints a list of notable findings from the ECG on the form that is later reviewed and confirmed or overturned by a licensed provider. In many cases, the machine interpretation is correct and is confirmed by the provider.",Operation and Maintenance,Both,2/1/2022,4/1/2022,10/1/2022,,Developed in-house.,,Yes,Veterans Health Administration,,No,This is a commercial OTC system that has been validated in clinical use over a period of years. It is compliant w/ VA OIT guidelines as it does operate on VA's network and interfaces with CPRS.,Yes,Yes,Other,The AI algorithm is validated by the manufacturer before the product is approved for use.,none,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ","[""Sex/Gender"",""Age""]",,No,No – agency does not have access to source code.,,Yes,Macview 360,Less than 6 months,No,,No,,Other,Unit is OTS ready. No provisioning is needed.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Appointment Comments Categorization,Department of Veterans Affairs,VA,OIT: Office of Information & Technology,Health & Medical,,None of the above.,To deliver additional timely options for patients that are having mental or physical issues during appointment signup.,"Right now, I'm focusing on alerting around mental health, so the model will return, ""Timely_Alert_Needed, 0.xxxxx""",Acquisition and/or Development,Safety-impacting,8/1/2024,8/1/2024,,,Developed in-house.,,Yes,Veterans Health Administration,,No," We are starting that process now.",Yes,No,Yes,," Comment data in cdw prod","Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Summit,Less than 6 months,Yes,,Other," Working with Summit now.",No,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, TeraRecon,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf22/K220349.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Ambient AI scribe,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,Transcribing and summarizing a recorded meeting or interview using AI.,"Human scribes have been effective toward decreasing clinical documentation burden, increasing physician satisfaction, and decreasing physician time spent in the EHR during patient visits. However, notes written by humans are variable in quality. In addition, scribes are costly, requiring time to train and have high turnover. Ambient AI medical scribes and solutions to automate document processing show promising as potential solutions to reduce physician burnout. VA recently completed an AI Tech Sprint in this space and intends to pilot several of these solutions at VA Medical Centers. ",The system produces written summaries of clinical encounters,Initiated,Both,5/21/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Pension Optimization Initiative (POI),Department of Veterans Affairs,VA,VBA: Veterans Benefits Administration,Government Services (includes Benefits and Service Delivery),,None of the above.,"VBA’s POI is a transformative effort that seeks a Managed Services Provider (MSP) to convert an existing manual business process into automated processing of Pension and Survivor claims. The goal is to have 75% of claims completed by automation with claims processing quality that is equivalent to, or greater than, existing manual processes. ","The following is high-level scenarios describe the inputs --->automated processes--->outputs: Scenario A: Claimant submits application for benefits via handwritten, typed, or VA.gov submission ---> POI automation extracts data from forms and other documents submitted with application ---> extracted data and existing Veteran/Beneficiary file data is processed in accordance with business rules to determine outcome of claim ---> outputs are: -updated Veteran/Beneficiary files, -completed Pension-related claim actions to include stop and start of awards (delivery of benefits), and -Notification Letters to Claimants. Scenario B: same as above, except, claim is established via computer matching agreement with other Government agencies (e.g. Social Security Administration) or other VBA-internal process rather than actually receiving an application from a claimant.",Operation and Maintenance,"Rights-Impacting ",4/1/2021,9/12/2022,1/27/2023,,,"36C10E19D0018, 36C10D22N0008",Yes,Veterans Benefits Administration,,No,Not applicable.,Yes,Yes,Other,Not sure.,Not sure.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.","[""Other""]",Not sure.,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,"VBA Automation Platform, VASI ID #2522",Less than 6 months,Other,Not sure.,Other,Not sure.,Other,Not sure.,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Limited Use of Azure Speech Services in PETALS Platform,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,"We have specific and limited use of Azure Speech Services in PETALS Platform, where Azure Speech Services will read either (a) a participant first name or (b) a numerical step count, and then read-aloud that content to validate the user and information input provided by user during program engagement. ",Audio verbalization (speak-aloud during telephone engagement) of a first name and of a numerical step count.,Acquisition and/or Development,Neither,6/14/2021,11/17/2023,,,Developed with both contracting and in-house resources.,36C250P2111,Yes,Veterans Health Administration,,No,n/a,Yes,Yes,No,,Training is not applicable in this case given that the 'data' used is provided to the system. Extensive testing has been done to validate/evaluate accuracy of speech services during extensive testing using test data.,"Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",,NONE,No,No – agency does not have access to source code.,n/a,Yes,Patient Engagement Tracking and Longterm Support and PETALS CS (Communication Services) - (eMASS 1341 and 2589),6-12 months,Yes,,No,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Medallia SaaS - VSignals and ESignals,Department of Veterans Affairs,VA,VEO: Veterans Experience Office,Other,Medallia is a customer experience management platform which can leverage AI (NLP) to review survey comments for actioning.,None of the above.,"Medallia is a customer experience management platform which can leverage AI (NLP) to review survey comments for actioning. VSignal survey comments (from Veterans) are processed by NLP to HELP determine if a Veteran is potentially in risk of crisis (related to homelessness and mental health). If NLP flags a comment as a potential risk, a case is created and sent to the Crisis Line and Homeless Program for review and actioning.",See above,Operation and Maintenance,Safety-impacting,1/1/2017,6/1/2017,1/1/2018,,Developed with both contracting and in-house resources.,VA-22-00077985 this is the PIID for Call Order 25 ,No,,,No,n/a,Yes,Yes,Yes,,Data from VHA's corporate data warehouse,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ","[""Race/Ethnicity"",""Sex/Gender"",""Age"",""Other""]",rural - urban,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Medallia eCloud-e,,Other,The infrastructure was available in 2017 so unclear if it went thru centralized intake,Other,the use case utilizes SAAS cloud product so not sure if the question is applicable,Other,see above,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, AGFA Dose Monitor system,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf18/K180589.pdf).,Operation and Maintenance,Both,12/28/2023,12/28/2023,12/28/2023,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Volpara Imaging Patient Hub,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf21/K211279.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Identity Governance and Administration (IGA),Department of Veterans Affairs,VA,OIT: Office of Information & Technology,Mission-Enabling (internal agency support),,Identifying unusual patterns in system logs from a single incident report using AI.,"The purpose and expected benefits of the AI are to reduce risks within VA workforce regarding access management. For example, the IGA tool will generate risk scores for access requests, allowing automation to approve low risk requests automatically while flagging others that are high-risk, sensitive, or critical for manual review. ","The system will display risk scores on a dashboard, send email notifications about access requests needing to be reviewed, or recommend access certifications/reviews.",Acquisition and/or Development,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, VET-HOME Contact Center AI ,Department of Veterans Affairs,VA,OIT: Office of Information & Technology,Health & Medical,,Searching for information using AI.,We're looking to explore the use of AI to give VA customer service agents better situational awareness to a Veteran through the use of AI tools to summarize and contextualize queries received through our contact centers. ,"We'd like to train custom co-pilots on data used by the contact center CRMs, to be able to proactively pull up information relevant to the conversation and the Veteran's queries. The output would be a window that would display information relevant to the Veterans needs at that time based on the contact center they are working with.",Initiated,Neither,9/23/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Beckam Coulter DxH 800,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf12/K120771.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, UIPath Document Understanding,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Mission-Enabling (internal agency support),,None of the above.,The tool enables Optical Character Recognition and data extraction from submitted documents through AI document processing. This tool can extract data accurately and efficiently to allow automations to input or use the data for other applications. ,"UiPath's Document Understanding will provide computerized information from forms and documents (computer generated or handwritten) that can be used by the automation to add information to other applications, or present to assist in decisions by the Agents for Veterans.",Operation and Maintenance,Neither,7/23/2023,7/23/2023,11/15/2023,,Developed with contracting resources.,"36C10B23F0307, NNG15SD66B",No,,,,,,,,,,,,,,,,Yes,VA Robotic Process Automation Platform 2755,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Potential Fraud or Waste,Department of Veterans Affairs,VA,OM: Office of Management,Mission-Enabling (internal agency support),,None of the above.,"The Purchase Card dashboards are a suite of dashboards that allow various purchase card managers to monitor purchase card spending of employees. The dashboards help monitor for fraud, waste, and abuse, and assist in providing oversight for compliance with purchase card laws and policies. We have now incorporated advanced data analytics within the dashboard. The Advanced data Analytics model generates an expected received date for items. It then flags items that have gone beyond the expected generated date by flagging items that were received late (possible waste) and/or items that have not been received yet (possible fraud). This has the potential to save the analyst time by investigating cases that have been flagged as opposed to investigating potentially an entire dataset, which is what they have been doing up until now. The model analyzes all purchase card transactions including ordered, delivery, and recieved dates as its inputs. It outputs expected recieved dates. The intended users of the product are purhase card managers. ",The Advanced data Analytics model generates an expected received date for items. It then flags items that have gone beyond the expected generated date by flagging items that were received late (possible waste) and/or items that have not been received yet (possible fraud).,Initiated,Neither,4/1/2024,,,,Developed in-house.,,No,,,No,N/A,No,No,Yes,,Purchase card transaction details,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ","[""Sex/Gender"",""Other""]",No demographic information is used.,Yes,"Yes – agency has access to source code, but it is not public.",,No,,6-12 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Roche Digital Pathology ,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,To aid pathologists in diagnosing pathology cases.,It analyzes slide images to aid in diagnosing,Acquisition and/or Development,Both,4/23/2024,12/15/2022,,,Developed with contracting resources.,GS-07F-0446V,Yes,Veterans Health Administration,,No,,Yes,Yes,No,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ","[""Other""]",None.,No,,,No,Navify,Less than 6 months,Yes,,No,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Activity recognition using wearable sensors for use in closed loop deep brain stimulation systems,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,"To accurately recognize/classify a number of different activities of daily living (walking, standing, sitting, lying down, etc.) using accelerometry/gyroscopic data. This could be used in a number of different applications. ",Described above. A classifier that outputs a recognized behavior in real-time based on changes in accelerometry/gyroscopic data.,Acquisition and/or Development,Safety-impacting,8/1/2019,12/16/2019,,,Developed with both contracting and in-house resources.,,Yes,Veterans Health Administration,,Yes,"Data privacy and confidentiality are ensured using standard techniques such as using a unique identifier for each research participant while keeping PHI in a separate spreadsheet kept on a VA-secured research server. For transparency, the underlying de-identified data will be uploaded to an online repository, the NIH AgingBioBank data repository. The code used for the AI use case is uploaded on Github.",Yes,No,No,,researcher-acquired accelerometer/gyroscope data,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,Yes – source code is publicly available.,https://github.com/alino93/HARDataset_LSTM,No,,,No,,No,,Other,"we have mostly used University resources for this, not VA. VA participants have been included in the data acquisition process, but no real VA resources have been used in model development.",None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, VCA & PPMS Chatbot ,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Mission-Enabling (internal agency support),,Searching for information using AI.,"The Network Support VCA & PPMS ChatBot, a virtual agent designed to assist field staff with questions related to VCA & PPMS. This tool was designed to provide answers to frequently asked questions received from the field.","The VCA and PPMS teams have compiled a wide range of FAQs received from field staff to develop this easy-to-use tool that instantly provides users with the VCA & PPMS resources they need, whenever they might need them.",Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, AgileMD eCART Clinical Deterioration Model,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf23/K233253.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, PINGOO.AI,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,Edcuational infomrational app that uses a RAG model to provide up to date and trustrowthy infomration to Veterans who are diabetic. ,Output is infomation that is related to the RAG model communicated to the Veteran through a language model interface.,Acquisition and/or Development,Both,9/26/2024,9/26/2024,,,Developed with contracting resources.,N/A,Yes,Veterans Health Administration,,Yes,RAG model verified and peer reviewed,No,No,Yes,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,6-12 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Persyst 14 EEG Review And Analysis Software,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,"Persyst EEG Review and Analysis Software provides the complete set of tools needed for C.A.R.E (Computer Assisted Review of EEG), resulting in accurate, efficient and rapid review of EEG data.Link to the vendors website - https://www.persyst.com/","It uses prespecified but customizable parameters for seizure detection, spike detection, trending of EEG signals, artifact reduction and monitoring. Its a commercial off the shelf product used widely in the field as well as public and private hospitals. ",Operation and Maintenance,Both,2/1/2024,,,,Developed with contracting resources.,"506-A40115, 36F79718D0330",Yes,Veterans Health Administration,,No," N/A",Yes,Yes,Other," This is a COTS system that exists at multiple VHA facilities for the purposes of reading EEGs and measuring seizure burden. "," N/A - COTS product with pre-specified parameters used for EEG analysis. ","Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",Other," None - Just uses EEG signals ",No,No – agency does not have access to source code.,,No,,Less than 6 months,Other," COTS Product that is installed on the National Tele-EEG Server. Uses pre-specified parameters. This is not true-AI. All information is available at vendor website. ",Other," not applicable",Other," not applicable",None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Computer Aided Detection (CADe) of Neoplasia during Colonoscopy - “GI Genius”,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,"GI Genius™, a commercial product distributed by Medtronic, Inc., is a computer-aided detection module, powered by artificial intelligence, designed to aid endoscopists in detecting colonic mucosal lesions (such as polyps and adenomas) during colonoscopy. The FDA cleared this commercial product in April 2021. The device physically connects to existing endoscopes, video processors, and display monitors, but does not connect to the VA network. During endoscopy, the CADe device automatically detects, and highlights suspected neoplastic lesions / polyps in real time. The module was trained and validated with white-light endoscopy videos and continues to be refined with growing datasets. Information about the indications, safety and warnings can be found here: GI Genius™ Intelligent Endoscopy Module - Indications, Safety, and Warnings | Medtronic (https://www.medtronic.com/covidien/en-us/products/gastrointestinal-artificial-intelligence/gi-genius-intelligent-endoscopy/indications-safety-warnings.html) Multiple studies and meta-analyses have found these devices increase adenoma detection rates, a well-established metric for colonoscopy quality that is related to colon cancer incidence and death. Initial randomized implementation of the devices across VA facilities allowed for a pragmatic evaluation of the impact of these devices on adenoma detection. The evaluation demonstrated that the provision of colonoscopy CADe devices resulted in a statistically significant 21% increase in the odds of adenoma detection and an absolute increase in ADR of approximately 4% compared to colonoscopy without CADe. These devices will improve ADR, which should decrease colorectal cancer incidence and mortality amongst Veterans over time. ","The device physically connects to existing endoscopes, video processors, and display monitors, but does not connect to the VA network. During endoscopy, the CADe device automatically detects, and highlights suspected neoplastic lesions / polyps in real time on the display monitor.",Operation and Maintenance,Both,6/1/2022,6/30/2022,10/12/2023,,Developed with contracting resources.,36C10X22X0045,Yes,Veterans Health Administration,,No,Not Applicable. This device does not disseminate information to the public; nor does it store or disseminate any information at all.,No,No,Other,Not applicable. No data catalog or repository required.,"No agency-owned data have been, nor are being, used to train the device. However, initial output/outcome evaluations of the device were conducted. The impact of CADe on adenoma detection rate (ADR) was not moderated by patient race, ethnicity, gender, or rurality (p>.05 for each); however, it was moderated by patient age (OR 1.02, 95% CI 1.00-1.03, p=.02), indicating that improvement in ADR after CADe availability was greater for older patients. The impact of CADe on ADR was moderated by the number of years since an endoscopist’s medical degree (OR 1.02; 95% CI 1.00-1.04, p =.04), indicating that ADR improvement with CADe availability was greater for endoscopists with more years since their medical degree. The impact of CADe on ADR was not moderated by endoscopist specialty or sex (p >.05 for both).","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.","[""Other""]","None. It does not have access to, nor use, any demographic variables.",No,No – agency does not have access to source code.,,No,,,No,,Other,"Not Applicable. This AI is a commercial, off the shelf, FDA cleared device.",Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,Yes,"Impact evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution and has utilized randomized experiments with a control group or other counterfactual, social systems analysis, or other rigorous research methodologies to evaluate impact and identify potential harm to users as well as broader groups of people.","Possible risk of increased false positive detection of colorectal lesions leading to unnecessary biopsies or polypectomy. This could result in increased costs and potential complications. Risk of over-reliance on detection by the physician (i.e., automation bias) Risk to education / training of fellows who may become reliant on the technology.",TRUE,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,Other,,No,"This FDA cleared AI device was formally evaluated by VA specifically looking for evidence of disparities in performance. The AI device was equally beneficial for all demographic groups evaluated, including by sex, race, and ethnicity. There was a modest increase in benefit with increasing patient age, which may be related to the increased prevalence of precancerous polyps with age.","[""Direct user testing"",""Post-transaction customer feedback collections""]",,No – it is not operationally practical to offer this.,This FDA cleared device is designed to assist the endoscopist in recognizing potential colorectal pathology; it does not make a diagnosis. It is ultimately the physician’s decision whether to act upon the findings highlighted by the AI device. The physician has full discretion in these actions. The physician can also turn off the device. There is no mandate to use this device.,Yes, Digizens,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,Behavioral prediction model used to create clinical trials without the need for actual patients or patient information. ,Output is behavioral analysis and predictions based on messaging and circumstances.,Acquisition and/or Development,Neither,9/26/2024,9/26/2024,,,Developed with contracting resources.,N/A,Yes,Veterans Health Administration,,No,N/A,No,No,Yes,,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,6-12 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, TrueFidelity CT Deep Learning Image Reconstruction,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,Decreases noise in CT images ,,Operation and Maintenance,Both,4/11/2019,4/11/2019,4/11/2019,,,,,,,,,,Yes,,,,,,,,,,Yes,MD-LITE ID: 2278,,,,,,,,,,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Pangaea,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,Devellop predictive model for chronic disease with treatment recommendations based on current guidelines. ,Output will be identification of chronic disease in Veterans and treatment modalities required prior to consulting specialty services.,Acquisition and/or Development,Both,9/26/2024,9/26/2024,,,Developed with contracting resources.,N/A,Yes,Veterans Health Administration,,No,N/A,No,No,Yes,,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,Yes,"Yes – agency has access to source code, but it is not public.",N/A,No,N/A,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, AIDOC,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf23/K230020.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Cogitativo,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,Predict chronic disease,Output is based on Veteran profiles utilizing data model built from previously obtained Veteran profiles by the company Cogitativo. It is unclear how the company obtained the data.,Acquisition and/or Development,Both,9/26/2024,9/26/2024,,,Developed with contracting resources.,none,Yes,Veterans Health Administration,,No,"Unclear, transparency was identified as an issue when going through gevernance.",Yes,No,No,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,No – agency does not have access to source code.,,No,,6-12 months,No,,No,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, "Analytics, Data, and Decision Support Unified Platform (ADDSUP)",Department of Veterans Affairs,VA,"OALC: Office of Acquisitions Logistics, and Construction",Other,Data Analysis,None of the above.,"Using AI enhancements, this tool can quickly filter contracts and line items to facilitate easy searching, as well as provide quick answers to enhance our ability to obtain cost avoidance. ",The system will output a list of contracts or line items based on search criteria entered by the user.,Operation and Maintenance,Neither,10/1/2023,1/1/2022,10/1/2023,,Developed with contracting resources.,36C10B21F0353,No,,,No,"ADDSUP is not subject to the Information Quality Act guidelines or OMB M-19-15 requirements because it does not disseminate information to the public, which is the primary focus of these federal directives.",No,No,Yes,,"VA owned data comes from eCMS, FPDS and D2D.","Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,No – agency does not have access to source code.,,No,,6-12 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, iCAD ProFound AI,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf20/K203822.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, VA Section 508 Office URL Ownership Prediction Model,Department of Veterans Affairs,VA,OIT: Office of Information & Technology,Mission-Enabling (internal agency support),,None of the above.,"Purpose of the AI system is to predict the administrative agency that owns URLs. The AI system reduces the level of effort and turnaround time for the OIT 508 Office to curate URLs and coordinate with VA Administrations to review and, as necessary, remediate URLs for accessibility compliance. In the absence of the AI system both the 508 Office and VA Administrations would be required to undertake a lengthy and iterative process to identify the VA Administration that is the formal owner of a URL.","The AI Model has two outputs. 1) Agency Owner with responses as: VHA, VBA, NCA, VACO, OIT, OCTO, and Unknown. 2) Prediction Score ranging from 0 to 1. The system is comprised of manually executed scripts using the AI model on local GFE. Besides the AI model, the scripts additionally apply manually constructed business rules and create prepopulated excel files for each VA Administration to use for quarterly reviews and data entry.",Operation and Maintenance,Neither,11/1/2023,11/1/2023,11/13/2023,,Developed in-house.,,No,,,No,,No,No,Other,In conjunction with the 508 Office we have confirmed that a comparable data set does not exist to with URL ownership data.,The data of available URLs is obtained from Google Analytics as part of the GSA Digital Analytics Program. Data to train the model is based on SME input from the VA administrations as part of their review process.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,"Yes – agency has access to source code, but it is not public.",,No,None. The system is GFE operated by OIT staff member providing consultative support to 508 Office.,Less than 6 months,No,,No,,Other,VA OIT(CRR)(DAPR)(DAPM) is the office providing support to the VA(OIT)(CRR)(RR)(508 Office). DAPM relies exclusively upon GFE and locally installed tools in the absence of pending authorization to access AI infrastructure in the SDP.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Circle CVI42 ,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,This software is for improving efficiency in post processing and reporting of Cardiac MRI and Cardiac CT images. ,"It replaces manual contouring of different cardiac structures by automated segmentation. Output includes: ejection fraction, strain, stroke volume , coronary calcium score etc. All outputs of the AI based software are FULLY CHECKED by the cardiologists for accuracy and the outputs can be easily altered by the reading physician before they are reported in the patients chart.",Operation and Maintenance,Neither,6/1/2022,1/1/2023,10/1/2024,,,,,,,,,,,,,,,,,,,,Yes,MD-LITE ID: 2278,,,,,,,,,,,,,,,,,,,,,,,,,,, National Cemetery Administration (NCA) Automated Claims Processing,Department of Veterans Affairs,VA,NCA: National Cemetery Administration,Government Services (includes Benefits and Service Delivery),,Inputting large amounts of data from paper forms into a digital system using AI.,"The purpose of the AI is to automate aspects of NCA benefits and application processing. Currently it is applicable to the Pre-Need and Presidential Memorial Certificate programs. In the future other business lines may be included, such as the Headstones, Markers, Medallions, and Urns and Plaques programs. This benefits NCA by automating aspects of the workload (data entry, case creation, document digitization, letter creation, status updates). ","The Hyperscience platform (optical character recognition (OCR), natural language processing (NLP), and machine learning (ML)) extracts document data, uses application programming interfaces (APIs) to query data from other VA systems, and applies robotic process automation (RPA) to enter data into existing legacy NCA systems: Eligibility Office Automation System and Web Presidential Memorial Certificates. As the case is processed, automation creates documents that combine relevant claimant information for use by claim agents, updates statuses, and creates letters as appropriate. No original content is created through these processes.",Operation and Maintenance,"Rights-Impacting ",4/1/2023,5/1/2023,8/14/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, VA CART Adenoma Detection NLP,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above., Extracts adenoma status for CART Veterans using Natural Language Processing (NLP) of VistA colonoscopy surgical pathology reports,Results contain 0 for no evidence of adenoma or 1 for evidence of adenoma in the pathology report,Operation and Maintenance,Neither,1/8/2024,5/31/2024,7/31/2024,,Developed in-house.,,Yes,Veterans Health Administration,,No,,Yes,Yes,Yes,,VistA colonoscopy surgical pathology reports,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,VistA colonoscopy surgical pathology reports,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,"Clinical Assessment, Reporting and Tracking Application (VASI ID 1079, EMAS ID 1808) ",6-12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Call Center Knowledge Navigator,Department of Veterans Affairs,VA,VBA: Veterans Benefits Administration,Mission-Enabling (internal agency support),,None of the above.,"Using VA-provided procedural reference material, the tool will respond to call center agent questions with an answer and a cited reference to verify the answer. The tool is intended to speed responses by quickly identifying relevant references and summarizing procedures in the form of an answer, while also providing the agent with the means to look up the referenced material.",The system will output a generated response to a user's question.,Implementation and Assessment,Neither,4/29/2024,5/9/2024,11/11/2024,,Developed with contracting resources.,47QTCK18D0036,Yes,Veterans Benefits Administration,,No,N/A; information is not accessible by the public.,No,Yes,Yes,,"The model uses data obtained from Education Service procedural resources (e.g., the School Certifying Official Handbook) to provide its responses.","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,No – agency does not have access to source code.,,No,,6-12 months,Yes,,Other,Resources through Azure have not been insufficient — implementation may show additional need.,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, National Training Team | Schools — FAQ Dashboard,Department of Veterans Affairs,VA,VBA: Veterans Benefits Administration,Mission-Enabling (internal agency support),,None of the above.,"The 1.0 dashboard clusters thousands of submitted questions during a monthly Office Hours session with School Certifying Officials and presents a ""top ten"" list of associated categories that have the most questions.",Classification and sentiment score,Implementation and Assessment,Neither,4/12/2023,4/28/2023,10/24/2023,,Developed with contracting resources.,47QTCK18D0036,No,,,No,,No,No,Yes,,Question-sets are obtained by exporting them from the monthly office hours sessions in Adobe Connect. These files are used to train and as the model's inputs.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,No – agency does not have access to source code.,,No,,Less than 6 months,No,,Other,Additional computing resources have not been necessary for this model.,Other,N/A,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, National Training Team | Schools — FAQ Dashboard,Department of Veterans Affairs,VA,VBA: Veterans Benefits Administration,Mission-Enabling (internal agency support),,None of the above.,"The model should classify thousands of questions into categories, generate a sample question based on the category, and provide a suggested answer based on previous responses to similar questions.","Classification, list of questions, answers to questions",Acquisition and/or Development,Neither,10/24/2023,10/24/2023,,,Developed with contracting resources.,47QTCK18D0036,No,,,No,,No,Yes,Other,Data is provided via exports of questions from Adobe Connect Office Hours; answer data is provided from previous FAQ sessions conducted by the Training Staff.,,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,No – agency does not have access to source code.,,No,,Less than 6 months,Other,No resources from OIT have been necessary for this project; development is undertaken by the contractor.,Other,N/A,Other,N/A,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Audit of Service Connection Designations Associated with Prescriptions,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Government Services (includes Benefits and Service Delivery),,None of the above.,"Reduction in waste, fraud and abuse. ",Binary,Acquisition and/or Development,Neither,7/11/2011,8/1/2011,,,Developed in-house.,,Yes,Veterans Health Administration,,No,"The use case is not broadly disseminating information to the public, so i don't this this is applicable.",Yes,Yes,Yes,,The information to train\validate the model is obtainable from the Corporate Data Warehouse environment.,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ","[""Other""]",Service-Rated Disability Conditions,No,Yes – source code is publicly available.,https://h2o.ai/platform/ai-cloud/make/h2o/,Yes,Data and Analytical Support for Healthcare (DASH) - #2515,Less than 6 months,Other,The design of DASH's intake is still to be determined because pending ATO is pending. The intake will occur at the facility level.,Yes,,Other,Two requests to ARCHES for a cloud-based development space have not yielded any response from their intake team.,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, PsychCorpCenter,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,Searching for information using AI.,The program is used to do multiculturally sensitive scoring of cognitive scores to make diagnostic impressions more accurate.,The information processed from the input of the providers on a specific Veteran/patient and then a report is generated/processed from the information that was inputted.,Acquisition and/or Development,Both,,,,,,,,,,,,,,,,,,,,,,,No,,,,,,,,,,,,,,,,,,,,,,,,,,,, ReflexAI,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Education & Workforce,,None of the above.,"AI-powered simulations to provide nuanced, high-quality practice and real-time feedback to every individual undergoing training to serve as a responder on Veterans Crisis Line.","All information is accessible for training staff within the secure platform hosted on VA Enterprise Cloud. This data include simulation content and the instantaneous feedback across the scoring dimensions. This output includes insights on training effectiveness overall, as well as progress of individual trainees who are going through the training process.",Operation and Maintenance,Neither,9/9/2022,6/6/2023,5/10/2024,,Developed with both contracting and in-house resources.,36C10B23F0271,Yes,Veterans Health Administration,,No,N/A,No,Yes,Yes,,N/A,,"[""Other""]",This question does not apply to this use case,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,"Hosted in ETIL, moving to Arches",6-12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, "Prostate Cancer, Genetic Risk, and Equitable Screening Study (ProGRESS) - Prostate Cancer Risk Prediction Model",Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Other,Research,None of the above.,"This risk model is being used within the framework of a nationwide, VA funded randomized controlled trial to deliver a precision prostate cancer screening intervention to research participants. The model uses machine learning components (e.g., training and validation sets from MVP and other external sources, LASSO-regularized Cox model) and genetic, ancestry, and self-report family history of prostate cancer data to develop and validate a risk prediction model to help stratify new participants into three distinct prostate cancer risk categories (low risk, average risk, high risk). The risk classification is then included as part of a comprehensive report, including a full risk model report from a VA-approved genetic testing laboratory, a monogenic genetic variant report from a VA-approved genetic testing laboratory, and a summary sheet which provide general information and risk-specific recommendations surrounding prostate cancer screening. The report is shared with the participant and their VA healthcare providers as additional information to be used within the framework of shared decision-making regarding future prostate cancer screening. ","The risk model uses new participant data to output a risk classification for prostate cancer based on genetic, ancestry, and self-report family history of prostate cancer data (low risk, average risk, high risk). The classification is is then included as part of a comprehensive report that is delivered to the participant and their VA healthcare providers to be used within the framework of shared decision-making regarding future prostate cancer screening.",Acquisition and/or Development,Both,8/1/2023,3/1/2024,,,Developed with both contracting and in-house resources.,"36C24124D0061, 36C24124C0018",No,,,No,N/A,Yes,Yes,Other,Model was developed and validated by VA researchers using Million Veteran Program (MVP) data available only to approved researchers. Model translation to a genetic test and intervention materials are being developed by a VA-approved genetic testing laboratory.,Million Veteran Program (MVP) data,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ","[""Race/Ethnicity""]","Model is specific to prostate cancer, so is only applicable to persons of male sex (not a specific variable in the model). Race/ethnicity data is genetically inferred ancestry.",Yes,"Yes – agency has access to source code, but it is not public.","Code is not publicly available, but is stored within VA systems (Genisis).",No,,Less than 6 months,No,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, "Using AI to enhance/augment classification of reported patient safety events, and to proactively identify patient safety vulnerabilities and potential areas of improvement",Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Mission-Enabling (internal agency support),,None of the above.,This system allows statisticians to identify trends and anomalies to proactively determine patient safety areas of improvement by identifying any new type of safety event and any abnormal increase in frequency of events and stop them before they become widespread.,Augmented event type classifications,Initiated,Neither,8/5/2024,,,,Developed in-house.,,Yes,Veterans Health Administration,,No,Not applicable,No,No,Yes,,,,,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,No,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, App feedback model for NLP tasks,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,"Functional Overview: The objective is to utilize Natural Language Processing (NLP) with comment reviews for App feedback, specifically to identify named entities (NER), profanity, and stop words, and provide an automated approach to pre-processing and cleaning text for downstream analytics tools. System Overview: The system uses spaCy's en_core_web_sm model, an open-source software and model, to analyze text reviews from various sources, including external sources (Google and Apple stores) and internal sources (FeedbackUI and VA Mobile). The model provides an out-of-the-box approach for NLP tasks, including NER, profanity detection, and stop word removal. Data: The data sources include: Text reviews from the VA's mobile applications on the Google and Apple stores (external sources) Reviews from FeedbackUI (internal source, available in the OIA_MobileHealth database) Reviews from VA Mobile (internal source, available via CSV files on the Mobile VA's internal website) Users: The users of this system are likely the OCC Data Science Team, who are responsible for developing and maintaining the pipeline. Target Audience: The target audience mobile application developers and other internal stakeholders who may be interested in analyzing and understanding the sentiment and feedback from these reviews. ","The objective is to utilize Natural Language Processing (NLP) with comment reviews for App feedback, specifically to identify named entities (NER), profanity, and stop words, and provide an automated approach to pre-processing and cleaning text for downstream analytics tools.",Operation and Maintenance,Neither,12/1/2022,2/1/2024,4/26/2024,,Developed with contracting resources.,"Unable to identify in time, working with Contracting team to identify. Iron Bow (Prime) - InnoVet (Sub)",No,,,No,N/A,Yes,No,Yes,,"CDW -VA application feedback data (Ex. Share My Health Data - application download count, user review)","Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.","[""Other""]",N/A,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,SDP / HDAP - working with contract team to better identify system nomenclature this falls under,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, TERA Memorandum Automation,Department of Veterans Affairs,VA,OIT: Office of Information & Technology,Government Services (includes Benefits and Service Delivery),,Searching for information using AI.,"Toxic Exposure Risk Activity (TERA) Memorandum Automation utilizes natural language processing (NLP) based algorithms to provide recommended decisions to support completing the TERA Memorandum form, a form that is required to be completed for most Veteran Disability Claims. This functionality will allow approved Veteran Service Representatives (VSRs) to auto-populate certain questions within the TERA Memorandum form in VBMS using evidence stored in the Claims Evidence drastically reducing the amount of time needed to manually review documents in Claim Evidence (eFolder) and streamlines claim processing. The system currently leverages data from C&P’s Smart Search service and utilizes NLP-based algorithms to identify potential evidence to support recommended decisions on specific questions of the TERA memo. The specific output includes an open-text field response with confidence score and links to documents for manual review. The intended users of the system are VSRs who complete the TERA memo. ","Toxic Exposure Risk Activity (TERA) Memorandum Automation utilizes natural language processing (NLP) based algorithms to provide recommended decisions to support completing the TERA Memorandum form, a form that is required to be completed for most Veteran Disability Claims. This functionality will allow approved Veteran Service Representatives (VSRs) to auto-populate certain questions within the TERA Memorandum form in VBMS using evidence stored in the Claims Evidence drastically reducing the amount of time needed to manually review documents in Claim Evidence (eFolder) and streamlines claim processing. The system currently leverages data from C&P’s Smart Search service and utilizes NLP-based algorithms to identify potential evidence to support recommended decisions on specific questions of the TERA memo. The specific output includes an open-text field response with confidence score and links to documents for manual review. The intended users of the system are VSRs who complete the TERA memo.",Operation and Maintenance,Neither,1/19/2024,1/19/2024,11/3/2024,,,,,,,,,,,,,,,,,,,,Yes,BIP Compensation and Pension User Interface ID: 2049,,,,,,,,,,,,,,,,,,,,,,,,,,, XtractOne,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Law & Justice,,None of the above.,"The new WDS will allow the VA Police to centralize all the facilities WDS reporting to one place allowing the VA to use manpower properly, allow them to also have better eyes on the WDS, and also allow them to have Enterprise solutions to allow upper management to see what is going on through the SaaS solution. This solution will also allow the VA Police to send out reports or alerts to Upper management and VA Police so they can mobilize and report to where the issues are quicker, efficiently, and effectively. This will also allow the VA Police to lock down facilities easier than before where that was not possible without going to said doors and locking them physically. The WDS can do that for the VA Police now. The key benefits are having a fully modernized weapons detection system to help the VA Police better detect unauthorized weapons entering and exiting the VA Facilities allowing the VA Police to do their job more efficiently and effectively. This will also all the VA Police to use their manpower in proper means instead of wasting man hours and money.","Xtract One View is equipped with advanced analytical tools that process vast amounts of data to detect unusual patterns and potential threats before they escalate. Using machine learning algorithms, it continuously improves its concealed weapons detection capabilities, adapting to new types of risks and optimizing security protocols accordingly.",Acquisition and/or Development,Both,9/23/2024,,,,,Contract has not been awarded yet.,No,,,No,,No,,No,,,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.","[""Other""]",Unknown,Yes,No – agency does not have access to source code.,,No,,Less than 6 months,No,,No,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Using AI to weekly monitoring of any harms resulting from the use of newly implemented electronic health record system – FEHR across VA,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Mission-Enabling (internal agency support),,None of the above.,This AI based monitoring system will be used to inform and improve the EHR system and it's functionalities to prevent and mitigate any patient safety events.,Classification of harm categories from EHR,Initiated,Neither,10/3/2022,,,,Developed in-house.,,Yes,Veterans Health Administration,,No,Not Applicable,No,No,Yes,,,,,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,No,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Podimetrics,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,"Our involvement with Podimetrics is limited to receiving and processing escalation reports generated by their SmartMat system. Podimetrics’ platform uses its proprietary technology, to generate these reports. However, our role does not involve developing, deploying, or controlling AI systems. Instead, we serve as a conduit, ensuring that these escalation reports are shared with the appropriate clinicians to facilitate timely and informed care for Veterans.","Our interaction with the Podimetrics platform is strictly limited to data transfer and integration of the generated reports. We do not engage with their proprietary technology, or decision-making processes inherent in the Podimetrics platform. Our role is supportive and integration-focused, solely aimed at ensuring seamless communication and clinical follow-up based on the provided reports.",Operation and Maintenance,Both,12/1/2019,12/1/2019,12/1/2019,,Developed with contracting resources.,,No,,,No,,No,Yes,No,,,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,No,No – agency does not have access to source code.,,,,,Other," N/A ",Other," N/A ",,,,,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Thematic Analysis of Chief Resident in Quality and Patient Safety (CRQS) Capstone Projects Using BERT Modeling,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Mission-Enabling (internal agency support),,None of the above.,"The intended purpose of the AI is to perform a detailed thematic analysis of CRQS capstone projects to identify key themes and trends, with the expected benefit of enhancing the national curriculum for quality and patient safety in healthcare.","The AI system's outputs will be categorized themes, patterns, and trends derived from the analysis of CRQS capstone projects, which will inform curriculum development and improvement strategies.",Initiated,Neither,7/1/2024,,,,Developed in-house.,,No,,,No,N/A,No,No,No,,"The model used in this use case will be trained, fine-tuned, and evaluated using agency-owned data comprising Chief Resident in Quality and Patient Safety (CRQS) capstone project documents, which detail various quality improvement initiatives and patient safety measures undertaken at VA Medical Centers.","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,No – agency does not have access to source code.,,No,,Less than 6 months,No,,No,,Other,N/A,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, My HealtheVet VSignals Main Improvement Summary,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,"My HealtheVet is the VHA patient portal to access to refill medications, secure message with care teams, access medical records and view appointment information. To ensure Veterans are being provided the best patient portal experiences, My HealtheVet collects many surveys (over 26k a month). AI provides initial summarizing and grouping functions for open-text questions to identify trending issues and new features requested. Rather than having humans read each comment and manually sorting, AI enhances trend identification and resource prioritization.","Veteran feedback is stripped of PII and then sorts comments based on product area, summary is primarily based on how often key words are mentioned.",Operation and Maintenance,Neither,7/19/2022,9/6/2022,9/9/2022,,Developed with both contracting and in-house resources.,36C10X20D0007,Yes,Veterans Health Administration,,No,N/A,Yes,No,No,,"This is using open-sourced code requiring no training, essentially performance was evaluated if correct keys words were logged to provide useable outputs.","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Other,"BISL did not consider the Python script AI, as it is more of a key word identification program. No resources were required for model training and development.",Other,None required.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, ESD-Speech Sentiment and Analytics,Department of Veterans Affairs,VA,OIT: Office of Information & Technology,Mission-Enabling (internal agency support),,Transcribing and summarizing a recorded meeting or interview using AI.,Utilizing AI to analyze speech sentiment of internal customers (non-veterans) to improve quality of service at the Enterprise Service Desk.,Various reports and dashboards for ESD leadership and the Tier 1 vendor partner to assess quality of service. The AI component makes it possible to assess and analyze all contacts with the ESD as opposed to the ~2% of contacts we can assess today using manual approaches.,Initiated,Neither,3/30/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Privacy Act Automation Services - Disclosure.AI,Department of Veterans Affairs,VA,VBA: Veterans Benefits Administration,Government Services (includes Benefits and Service Delivery),,None of the above.,"The PAA System will automate PA workflows that are currently highly manual. Automating PA processing will benefit Veterans by decreasing the time they have to wait to receive requested records, and it will benefit the VA by reducing the time that staff need to support PA request processing. Tools/techniques that may be categorized as AI improve the automated processing that provide these benefits, but AI technology is ancillary to other technology utilized in the PAA system. Machine learning is used to identify specific documents (DD-214, Decision Letters, Service Records, etc.) and AI is used to improve and expedite data extraction as well as identify and apply automated redactions. The process is designed with Human in the Loop (HITL) and Government in the Loop (GITL) roles.","The PAA System automates the process of ingesting PA requests, retrieving requested records, and redacting requested records when necessary. The outputs of the PAA System are reviewed by Deloitte’s managed service team and by the VA for accuracy and completeness before the requested records are sent to the requester. The outputs consist of records held by VBA within Claim Evidence (VBMS) that have appropriate redactions applied for 3rd party information before release to the 1st party requestor or their designee.",Implementation and Assessment,Neither,1/10/2023,9/27/2023,11/18/2024,,Developed in-house.,,Yes,Veterans Benefits Administration,,No,"N/A. D.AI does not involve scientific, financial, or statistical information.",Yes,Yes,Yes,,"Agency Enterprise Data Warehouse (EDW) data is used to track pending and completed Privacy Act requests for information and their respective performance metrics, to include average days pending and average days to complete to ensure the D.AI solution exceeds the metrics for the existing manual process and meets the contractually obligated service levels for accuracy and turn around (completion) times.","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,No – agency does not have access to source code.,,Yes,Privacy Act Automation/Disclosure.AI,More than 12 months,No,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Sentiment analysis for app feedback,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,"Functional Overview: The objective is to perform sentiment analysis on text reviews to classify their content as positive, negative, or neutral, and to provide a processed dataset for further downstream analytics. System Overview: The system uses a trained model, derived from an open-source model and fine-tuned for the specific dataset, to analyze text reviews from various sources, including external sources (Google and Apple stores) and internal sources (FeedbackUI and VA Mobile). The model classifies reviews as positive, negative, or neutral. Data: The data sources include: Text reviews from the VA's mobile applications on the Google and Apple stores (external sources) Reviews from FeedbackUI (internal source, available in the OIA_MobileHealth database) Reviews from VA Mobile (internal source, available via CSV files on the Mobile VA's internal website) Users: The users of this system are likely the OCC Data Science Team, who are responsible for developing and maintaining the pipeline. Target Audience: The target audience OCC mobile application developers and internal stakeholders who may be interested in analyzing and understanding the sentiment and feedback from these reviews. ","perform sentiment analysis on text reviews to classify their content as positive, negative, or neutral, and to provide a processed dataset for further downstream analytics.",Operation and Maintenance,Neither,12/21/2022,3/8/2024,4/24/2024,,Developed with contracting resources.,"Unable to identify in time, working with contracting team to close",No,,,No,,Yes,No,Yes,,CDW - VA application text review commentary for sentiment analysis,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.","[""Other""]",,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,SDP / HDAP - working with contract team to identify most applicable ATO system name,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, ESD-Predictive Intelligence,Department of Veterans Affairs,VA,OIT: Office of Information & Technology,Mission-Enabling (internal agency support),,None of the above.,The intended purpose is to leverage Machine Learning on existing ticketing data supporting internal customers (non-veterans) to reduce mean time to resolution for internal customers while also increasing ticket routing accuracy and identifying major incidents more quickly as they occur.,Outputs involve pre-populated assignment groups in tickets within the ServiceNow IT Service Management platform to provide more accurate ticket routing and dashboard information for Incident Management staff to review highlighting potential major incidents beginning to occur in the environment.,Initiated,Neither,7/1/2024,,,,Developed in-house.,,No,,,No,,No,Yes,Yes,,Ticketing data in the ServiceNow IT Service Management platform.,"Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.","[""Other""]","""None"" was not an option, but the answer here is none.",No,"Yes – agency has access to source code, but it is not public.",,Yes,ServiceNow ,More than 12 months,Other,It's part of the ServiceNow IT Service Management platform,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, AI Health Coach,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,The Health Coach and AI components are different. The Health Coach helps improve adherence by sending evidence based nudges (this doesn't involve AI). The AI makes the health coach more effective by adjusting the number of nudges sent to the engagement of the Veteran.,"The Health Coach has a pre-selected decision tree (ex: did you miss your medication -> if yes, how many medications did you miss -> did you miss them because you ran out of refills). The AI simply helps determine how often to send those nudges based on response rates.",Operation and Maintenance,Neither,5/1/2024,9/22/2022,5/1/2024,,Developed with both contracting and in-house resources.,NG15SD26B,No,,,No,"The VHA has worked with the NAII, ISSO, Privacy, and VA Innovation Unit (OIT) through Angela Gant Curtis and coordinated with AI leads on efforts such as Kimberly McManus. After rigorous privacy, security, and safety review by clinicians in addition to the creation of standard operating procedures the AI Health Coach was put into place for Veterans who were matched based on agency data in a closed server. Information is not disseminated to the public via the AI Health Coach but only provided to Veterans actively engaged in primary care.",Yes,Yes,Yes,,"Master Patient Index (IEN, DOB, Address, Phone, PCMM)","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.","[""Other""]","(IEN, DOB, Address, Phone, PCMM)",No,"Yes – agency has access to source code, but it is not public.",,Yes,1366,More than 12 months,Yes,,Other,"We work directly with the contractor for model details. We assess efficacy based on clinical outcomes at this time, given that the overall program will be run for a total of one year.",Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Verathon Prime Plus,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,It's a bladder scanner. The AI component is used in relation with detection of bladder. It's aim is to increase accuracy.,It outputs assisted measurements related to bladder size and shape.,Operation and Maintenance,Both,1/1/2024,1/1/2018,1/1/2018,,Developed with both contracting and in-house resources.,,Yes,Veterans Health Administration,,No,,No,No,No,,,,"[""Other""]",None,No,No – agency does not have access to source code.,N/A,No,,Less than 6 months,No,,No,,No,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, App Feedback categorization model,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,"Functional Overview: The objective is to categorize review texts into specific categories to provide more granularity and enable users to identify trends and perform analytics on the data. System Overview: The system uses a trained open-source model that has been fine-tuned on the app feedback data to categorize review texts into specific categories, including Tech, Usability, Content, Idea, and Other. Data: The data sources include: Text reviews from the VA's mobile applications on the Google and Apple stores (external sources) Reviews from FeedbackUI (internal source, available in the OIA_MobileHealth database) Reviews from VA Mobile (internal source, available via CSV files on the Mobile VA's internal website) Users: The users of this system are likely the OCC Data Science Team, who are responsible for developing and maintaining the pipeline. ",The objective is to categorize review texts into specific categories to provide more granularity and enable users to identify trends and perform analytics on the data.,Operation and Maintenance,Neither,12/2/2023,3/1/2024,4/16/2024,,Developed with contracting resources.,"Unable to identify, working with contracting team to find most applicable PIID",No,,,No,,Yes,No,Yes,,CDW - VA application feedback,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,SDP / HDAP - working with contract team to better identify applicable system name,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, LINQ and LINQ II Loop Recorder ,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,Implantable heart monitoring device. This continuously checks the heart rhythm and rate. ,The information (recordings) are remotely transmitted and recorded for interpretation by a Medical provider.,Operation and Maintenance,Both,12/21/2021,12/7/2021,12/27/2021,,Developed with contracting resources.,Sara Lind Boston Scientific Peripheral Interventions Field Inventory Management sara.lind@bsci.com P: 763.255.0037,Yes,Veterans Health Administration,,No,N/A,Yes,Yes,No,,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ","[""Sex/Gender"",""Age""]",,No,No – agency does not have access to source code.,N/A,No,N/A,Less than 6 months,Yes,N/A,No,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, VA.gov Chatbot: Use of AI for summative and/or intent classification of Va.gov content,Department of Veterans Affairs,VA,OIT: Office of Information & Technology,Government Services (includes Benefits and Service Delivery),,None of the above.,"The VA.gov Chatbot currently uses Natural Language Understanding (primarily driven by Microsoft Power Virtual Agents (PVA) and Custom Question Answer (CQA)) to interpret Veterans' intent and match it to responses within the knowledge base. The future state vision for the VA’s chatbot solution involves leveraging advanced AI services and capabilities to increase the breadth of questions that the Chatbot will be able answer, to improve the accuracy of each answer, and in cases of necessary channel escalation, to accurately match user utterances to contact center routing queues.",Input: Understanding user intent: Copilot can determine what the user is trying to accomplish based on their query. Output: Generating relevant responses: The AI can then provide a suitable response or action based on the user's intent.,Operation and Maintenance,Neither,9/1/2021,9/1/2021,3/2/2022,,Developed with both contracting and in-house resources.,"VA118-16-D-1015,36C10B21N10150056",No,,,Yes,"The Chatbot is open to the public. In the above, we interpret ""Disseminate"" to refer to ability of the Chatbot to provide information that appears on VA.gov to users who ask general questions of it.",No,Yes,Yes,,Customer Experience Insights (CXI) data lake - all chatbot transcripts are written here and leveraged in analysis,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,VA.gov Chatbot,,No,,No,,Other,N/A,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, ZIO PATCH HEART MONITOR,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,Creating visual representations of data sets for reports and presentations using AI.,Records the heart rate and rhythm for up to 14 days to diagnose arrhythmias and any other electrical conduction issues.,N/A,Operation and Maintenance,Both,,,,,,,,,,,,,,,,,,,,,,,Yes,MD-LITE ID: 2278,,,,,,,,,,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Caller intent classification using Google Dialogflow (VoiceBot),Department of Veterans Affairs,VA,OIT: Office of Information & Technology,Government Services (includes Benefits and Service Delivery),,Searching for information using AI.,"This project is part of the Unified Communications goal of modernizing call centers. AI (as NLU/NLP) can expedite triage, improve caller-to-agent matching, and handle routine tasks (through APIs) without agent support. ","Mainly conversational output. The VoiceBot will act as a caller-concierge, helping Veterans locate the right agent and/or solve tasks (e.g., claims status checks). Conversational scripts, including intents, entities and slots will be stored in CXI and/or other databases (e.g., Clarabridge) standard to VA call centers today.",Acquisition and/or Development,Safety-impacting,7/1/2022,12/1/2022,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Ordr,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,Searching for information using AI.,Ordr SCE Analytics employs advanced artificial intelligence and machine learning (AI/ML) technologies to further assist in the classification of local networked devices required for accurate device inventory. AI/ML enhances and accelerates the classification process by identifying specific behavior patterns that are unique to certain devices. These passive classification techniques are minimally invasive and do not rely on active scanning methods which are often disruptive to sensitive business-critical IoT devices.,Only network device metadata is forwarded to the SCE Analytics server. Sensitive data such as PHI is immediately discarded so that only metadata which contains information pertinent to device identification is forwarded.,Operation and Maintenance,Neither,11/2/2021,3/7/2022,6/27/2024,,,,,,,,,,,,,,,,,,,,Yes,ORDR,,,,,,,,,,,,,,,,,,,,,,,,,,, Machine Algorithm for Report Surveillance (MARS),Department of Veterans Affairs,VA,OEHRM: Office of Electronic Health Record Modernization,Health & Medical,,Searching for information using AI.,The purpose of the MARS tool is to identify end-user submitted federal EHR incident reports that are likely to affect patient safety. The MARS tool will augment the existing manual incident report review process. ,The output of the system is a list of ServiceNow tickets that require human review to validate the MARS determination that the issue described in the ticket is a patient safety issue.,Initiated,Neither,9/1/2022,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Pharmacy AI Managed Inventory System (PHAIMIS),Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,"PHAIMIS will be an AI-powered inventory management system to centralize and optimize inventory processes within the VA Pharmacy Service. This system will integrate AI and data analytics to streamline inventory management, provide real-time notifications, and enhance user experience through seamless integration into existing workflows.","System outputs will include demand forecasting, procurement strategy optimization, and dynamic inventory thresholds. It may also include anomaly detection alerts and other features.",Initiated,Safety-impacting,5/21/2024,,,,Developed with both contracting and in-house resources.,,No,,,No,,No,No,Other,"We do not yet have access to ALL data, such as vendor supply data, but not all data is necessary to create a solution that provides value. We do have access to all data required for forecasting filled prescriptions, for example, and this alone provides value if the scope of the use case needs to be narrowed accordingly.",Prescription fill time-series data (note: PHI/PII not needed) needed for demand forecasting. Pharmacy inventory data needed for ordering recommendations.,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,,,Yes,Summit Data Platform,Less than 6 months,No,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Silverberry Surgery Planning AI,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,"Assist in planning and operations in the perioperative process including sps, supplies, personell requirements and patinet planning. ",Output will be multiple AI agents responsible for various tasks related to the peri operative process.,Acquisition and/or Development,Both,9/26/2024,9/26/2024,,,Developed with contracting resources.,N/A,Yes,Veterans Health Administration,,No,N/A,No,No,Yes,,None,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,N/A,Yes,"Yes – agency has access to source code, but it is not public.",N/A,No,N/A,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, ICU CIS/ARK PDF Medical Entity and Clinical Scoring Extraction (PHI 3.5 LLM),Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,"Extract clinical data elements from unstructured clinical note text into structured and semi-structured formats, mapping to standardized terminologies, thus making unstructured note text available to downstream analytics processes and clinical information systems.","Extracted data elements in structured and semi-structured formats, and mappings to standardized terminologies such as SNOMED, LOINC, ICD-10 and RxNorm.",Acquisition and/or Development,Safety-impacting,6/1/2023,6/3/2024,,,Developed in-house.,,No,,,No,,Yes,Yes,Yes,,"TIU notes, CISARK PDF documents and other unstructured and semi-structured text-based patient care records","Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Arches,Less than 6 months,No,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, BlueTeam AI,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Mission-Enabling (internal agency support),,Identifying unusual patterns in system logs from a single incident report using AI.,BlueTeam AI enables network monitoring capabilities to provide governance and guardrails for LLM Generative AI use for the enterprise.,The system outputs only approved data and data categories to LLM as well as outputs a return notification to the user for any violations.,Implementation and Assessment,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, "OAWP Investigative, Analytics, and Reporting Capabilities",Department of Veterans Affairs,VA,OAWP: Office of Accountability and Whistleblower Protection,Mission-Enabling (internal agency support),,Transcribing and summarizing a recorded meeting or interview using AI.,"We are in the initial phases of prototyping an audio transcription application using Azure AI Speech Services. The goal of this project is to provide a streamlined means for OAWP investigators to receive transcripts of subject interviews. Additionally, OAWP's Compliance Analytics and Reporting Division has long-term goals to use a supervised machine learning model to classify external oversight entity reports. They are also evaluating opportunities to use deep learning models to explore data, and identify trends and anomalies.","Enhanced Analytics, Reporting, and Transcription Services.",Initiated,Neither,6/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Multi-Modal Digital Image Exchange - AI (MDIE-AI),Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,It creates an AI driven orchestration layer and generative AI summary for automating computer vision models with saved dicom diagnostic images.,The system will output the results of each model into a generated summary PDF that is available for review directly from the dicom file.,Acquisition and/or Development,Safety-impacting,1/22/2024,2/19/2024,,,Developed in-house.,,No,,,No,maintain data within current ATO environments.,Yes,Yes,Yes,,"radiology images, reports, problem list, demographics","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.","[""Race/Ethnicity"",""Sex/Gender"",""Age""]",,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,ARCHES,6-12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, VA Supply Chain Knowledge Management Dashboard,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Mission-Enabling (internal agency support),,Searching for information using AI.,"The goal is to leverage already existing Power BI tools to enhance search and learning capabilities. The Power BI dashboards are for internal VALOR mission topics that require timely and more precise knowledge management (KM) and lessons learned output from publicly available documentation (ex. congressional reports). THIS CAPABILITY IS NOT RIGHTS OR SAFETY IMPACTING AS IT DOES NOT ACCESS ANY INDIVIDUALS INFORMATION.","This involves enabling a knowledge management strategy that includes advanced data analysis tools (AI and ML) from Power BI stakeholder engagement for validation, dynamic dashboards for knowledge sharing, and continuous improvement processes. This strategy ensures that knowledge is effectively captured, validated, shared, and utilized to drive continuous improvement and mitigate risks. The expected impact includes internal mission use by VALOR for improved risk mitigation, enhanced collaboration, and continuous process improvement. And the overall outputs are only data provided by the public documents.",Acquisition and/or Development,Neither,7/24/2024,8/2/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Clinical outcomes for asynchronous teledermatology,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,Determine clinical outcomes of patients with skin disease for research and QI purposes.,"Classify dermatology progress notes into 6 classes: Resolved, improved, unchanged (clinically significant), unchanged (not clinically significant), worse, and No outcome stated.",Acquisition and/or Development,Neither,6/9/2020,6/1/2022,,,Developed in-house.,,No,,,No,N/A,No,No,Yes,,Clinical notes and encounter data from CDW,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,Yes – source code is publicly available.,https://www.llama.com/,No,VINCI,Less than 6 months,No,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, VA TryOpenAI,Department of Veterans Affairs,VA,OIT: Office of Information & Technology,Mission-Enabling (internal agency support),,None of the above.,"TryOpenAI aims to improve VA employee efficiency and satisfaction. TryOpenAI can be used complete basic administrative tasks like drafting emails and communications, summarizing long text, and draft template project plans (among other similar uses). ","Tasks pilot users use TryOpenAI for Writing Enhancement and Communication Support: - Drafting and editing professional emails, letters of recommendation, and narrative write-ups. - Generating and refining performance appraisals, award write-ups, and administrative correspondence. - Summarizing large documents including meeting transcripts and project reports for email announcements Administrative Task Automation: - Streamlining the creation of meeting minutes to improve efficiency in documentation workflows. - Automating responses for customer service inquiries or generating template-based outputs for routine tasks. - Organizing thoughts and prioritizing information to aid in decision-making Creative and Technical Problem Solving: - Brainstorming ideas for project development and strategy formulation. - Assisting with technical problems such as understanding PowerShell scripting or SQL configuration settings. - Designing website layouts, drafting emails with complex content, and conceptualizing Power apps Research Aid and Learning Tool: - Gathering information on specific subjects like VA public information or medical guidelines. - Finding synonyms or generating ideas for naming conventions in research contexts. - Seeking deep insights into subject matter for personal knowledge enhancement or educational purposes Amongst other similar tasks",Implementation and Assessment,Neither,9/1/2023,10/1/2023,11/1/2023,,Developed with contracting resources.,"47QTCA22D003G, 36C10B22F0089",No,,,No,,Yes,Yes,No,,None,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,TryOpenAI,Less than 6 months,Yes,,Other,"We do not require computing resources for model training and development in this case. We DO require computing resources for model usage, though",Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Lyssn for Mental Health,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,"To improve fidelity to evidence based practices in Behavioral Health, such as Motivational Interviewing. ","Lyssn uses AI algorithms to automatically generate clinical quality metrics from a recording (or transcript) of a provider-patient session or appointment. Metrics range from general counseling and patient-centered communication to fidelity to evidence-based counseling, including Motivational Interviewing and Cognitive Behavioral Therapy.",Operation and Maintenance,Neither,10/3/2022,6/15/2023,9/5/2023,,,,Yes,Veterans Health Administration,,No,,No,No,No,,,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ","[""Ability Status""]",,No,No – agency does not have access to source code.,,No,,,No,,No,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, SOMATOM go.Up; SOMATOM go.Now; SOMATOM go.All; SOMATOM go.Top; SOMATOM go.Sim; SOMATOM go.Open Pro; SOMATOM X.cite; SOMATOM X.ceed,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf23/K233650.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, "HCD User Feedback Summary, Analysis, and Design",Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,Summarizing the key points of a lengthy report using AI.,"The Human-Centered Design (HCD) team uses Large Language Models (LLMs) and custom agents to enhance user experience (UX) research. The primary purpose is to improve the usability, accessibility, and clarity of user feedback, providing a better understanding of the needs of Veterans and healthcare providers.","The AI system generates insights, such as user stories, personas, and workshop summaries, which are used to inform design decisions.",Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,Yes,SDP ID: 1301,,,,,,,,,,,,,,,,,,,,,,,,,,, 3M/Solventum 360 Encompass Computer Assisted Coding - Auto Suggestion,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,Summarizing the key points of a lengthy report using AI.,"The intended purpose is to provide medical coder end users with suggested ICD 10, CPT, and HCPCS codes to select to be assigned/associated to an outpatient, inpatient or surgery encounter based on the provider's clinical documentation. The expected benefit for the medical coder to gain efficiency in the medical coding process that results in more accurate and timely coding of encounters.","Suggested ICD 10 CM, ICD 10 PCS, CPT, and HCPCPS codes to a medical coder end user based on the clinical documentation the AI system reads.",Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Vivid iq,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf22/K221148.pdf),Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Vivid E80/ Vivid E90/ Vivid E95,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf22/K220882.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Payment Redirect Fraud (PRF) Model,Department of Veterans Affairs,VA,VBA: Veterans Benefits Administration,Government Services (includes Benefits and Service Delivery),,None of the above.,"Criminals make direct deposit (DD) changes to steal Veterans’ benefit payments. Most direct deposit changes are safe, but 1-2 out of 1,000 (.1% - .2%) are fraudulent. The goal of the Payment Redirect Fraud (PRF) model is to identify which DD changes are likely to be fraudulent and refer them to team investigators for review and remediation.",The model identifies referrals (Veterans) with the highest scores indicating likely fraudulent activity. The outputs include Veteran PII and associated direct deposit bank account information. These outputs are in the form of an R data file and an Excel file.,Operation and Maintenance,"Rights-Impacting ",10/1/2019,12/1/2021,10/21/2022,,Developed with both contracting and in-house resources.,GS-00F-008DA,No,,,No,Results from the PRF model are for internal use only not shared outside of BDPR.,No,No,Yes,,We use the population data from all historical fraud cases as well as model learning as a result of daily outputs.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.","[""Income"",""Age"",""Other""]",,Yes,"Yes – agency has access to source code, but it is not public.",NA,No,"The PRF model does not run in a systems or a cloud, it runs by manual intervention on employee/contractor GFE machines",More than 12 months,No,,Yes,,No,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,Yes,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.","False positives could result from the model output; however, this is mitigated by the fact that all referrals from the model output undergo a manual investigation by a fraud investigator.",TRUE,"Automated and Regularly Scheduled Updates: Some aspects of the plan to monitor the AI system post-deployment are automated including re-training of models after detecting drift; however, data science teams are still significantly involved in the monitoring and re-deployment process.",No - Some individual decisions or actions require direct human oversight.,,N/A - individuals are not interacting with the AI for this use case,,No,The model does not incorporate race or gender into the model features.,"[""None of the above""]",,No – it is not operationally practical to offer this.,"It is not practical to offer this because final decisions of fraud are based on human investigation and not solely on the model output. The AI functionality is solely used to detect cases that are potentially fraudulent. All cases are independently reviewed by fraud investigators to validate whether they are, in fact, fraud.","No – This AI use case is not subject to the opt-out requirement because the AI functionality is solely used for the prevention, detection, and investigation of fraud or cybersecurity incidents, or the conduct of a criminal investigation.",NA "EchoPAC Software Only, EchoPAC Plug-In",Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf22/K220940.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Using Partially Observed Markov Decision Process to Implement a Response to Intervention Framework for Veterans with Post-Traumatic Stress Disorder ,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,The Partially Observed Markov Decision Process (POMDP) model will be eligible to help clinicians to provide more personalized treatment to minimize or eliminate the PTSD symptoms in a short amount of time (thus costing less) and enable veterans to return to civilian life more quickly than the current practice,,Initiated,Both,12/4/2023,,,,Developed in-house.,research IRB approved,No,,,No,The project is required to adhere to all the safety requirements governed by IRB,Yes,Yes,Other,Already identified,"labs, demographics, visits, problems, medications, social history","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ","[""Race/Ethnicity"",""Sex/Gender"",""Age"",""Socioeconomic Status"",""Ability Status"",""Residency Status"",""Marital Status"",""Employment Status""]",,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,VINCI,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, Microsoft Power Automate and AI ,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Transportation,,None of the above.,"To retrieve information from external facilities (SNF, CNH) in order to allow our facility (Veterans Transport Program) to facilitate scheduling of transport of eligible veterans to and from our facility or to community care appointments.","The AI program extracts information from a request form (pdf) attached to an email. The information is then populated in an Excel spreadsheet. The data points include the date of request, date transport needed, name of requesting facility, person completing the form, Veteran's last name, first name and last 4, attendance needed, trip type, pick up time, appointment time, facility pick up address, appt location, destination address, oxygen needed, weight of patient.",Operation and Maintenance,Neither,7/5/2023,5/1/2023,6/1/2023,,,,Yes,Veterans Health Administration,,No,,Yes,Yes,No,,,,,,No,,,No,MS Office 365 - Power Automate ,Less than 6 months,No,,No,,Other,Not applicable,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, IVC Access Transformation Transformative Care Modalities VA Health Connect Using Machine Learning to Evaluate First Contact Resolution (FCR),Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,For analytics purposes to generate performance metrics on First Contact Resolution for the Clinical Triage core service. Does not influence medical or critical decisions and does not use any sensitive Veteran data.,"The output of the model allows us to classify each clinical triage as resolved or not resolved, so that we can generate performance metrics on First Contact Resolution for the Clinical Triage core service.",Acquisition and/or Development,Neither,10/23/2023,10/24/2023,,,Developed with contracting resources.,"VA118-16-D-1015, 36C10B21N10150056",No,,,No,N/A,No,No,Yes,,The language model is being used to compare free text fields within the Customer Relationship Management (CRM) application that capture details about the purpose of each case.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,N/A,Yes,"Yes – agency has access to source code, but it is not public.",N/A,Yes,Summit Data Platform,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, ECG system software,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,Creating visual representations of data sets for reports and presentations using AI.,Commercially available software to assist with electrocardiogram interpretation,Produces an ECG tracing and provides reading on ECG parameters,Operation and Maintenance,Both,,,,,,,,,,,,,,,,,,,,,,,Yes,MD-LITE ID: 2278,,,,,,,,,,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Avicenna ICH ,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,The function of the AI is to detect possible intracranial hemorrhage (ICH) on CT head exams to aid and expedite diagnosis.,The input is DICOM format CT head image data from VHA facilities that submit the exam to VHA National Teleradiology Program (NTP) for interpretation. The output is DICOM object written to the NTP Picture Archive Communication System (PACS) for review by an interpreting VHA NTP Diagnostic Radiologist. Future output will also include a DICOM structured report to the VHA NTP PACS Powerscribe dictation software.,Operation and Maintenance,Both,4/1/2020,6/1/2020,10/1/2021,,Developed with contracting resources.,36C10B19C0016,Yes,Veterans Health Administration,,No,N/A,Yes,Yes,Yes,,"N/A - FDA 510K cleared COTS product, trained using OEM data","Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.","[""Age""]",,No,No – agency does not have access to source code.,N/A,Yes,National Teleradiology Program Next Generation Picture Archive and Communication System #2418,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Densitas Density AI,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,"The function of the AI is to incorporate quantitative mammographic density with the qualitative 4-category (A, B, C, D) breast density scale that aligns with the ACR BI-RADS Atlas 5th edition breast density scale. The AI accounts for both the degree of mammographic density and the presence of localized areas of dense breast tissue to aid and expedite mammographic diagnosis.",The input is DICOM mammography image data from VHA facilities that submit digital mammogram exams to VHA National Teleradiology Program (NTP) for interpretation. The output is DICOM object written to the NTP Picture Archive Communication System (PACS) for review by an interpreting VHA NTP Diagnostic Radiologist. Future output will also include a DICOM structured report to the VHA NTP PACS Powerscribe dictation software.,Operation and Maintenance,Both,6/1/2022,9/1/2022,11/1/2022,,Developed with contracting resources.,36C10B22C0042,Yes,Veterans Health Administration,,No,N/A,Yes,Yes,Yes,,"N/A - FDA 510K cleared COTS product, trained using OEM data","Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.","[""Age"",""Sex/Gender""]",,No,No – agency does not have access to source code.,N/A,Yes,National Teleradiology Program Next Generation Picture Archive and Communication System #2418,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, CLARUS,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf19/K191194.pdf).,Operation and Maintenance,Both,2/4/2022,2/4/2022,2/4/2022,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Riverain ClearRead CT,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,"The function of the AI is to analyze CT chest exams and support identification and characterization of pulmonary nodules of all types (sold, part-solid, ground glass), especially for lung cancer screening exams. The CT Vessel Suppress produces a secondary series of CT images, suppressing vessels and other normal structures within the lungs to improve conspicuity of pulmonary nodules. ClearRead CT Compare extends detection by automatically matching nodules found in the current exam to the same nodule on a prior exam. The AI is managed by VHA National Teleradiology Program (NTP) and is hosted in VA Enterprise Cloud (VAEC) or on-premise servers depending on the individual VISN/facility. ","The input is DICOM CT chest image data. The output is DICOM object written to the PACS for review by an interpreting Diagnostic Radiologist. Future output will also include a DICOM structured report to the voice recognition dictation software. The AI is deployed by NTP for the use of the individual VISN and facility Radiologists interpreting the exam, or for NTP Radiologist use.",Operation and Maintenance,Both,6/1/2021,9/1/2022,2/1/2023,,Developed with contracting resources.,36H79719D0016,Yes,Veterans Health Administration,,No,N/A,Yes,Yes,Yes,,"N/A - FDA 510K cleared COTS product, trained using OEM data","Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.","[""Age""]",,No,No – agency does not have access to source code.,N/A,Yes,VHA Centralized Lung CT CAD System #2219,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Clinical Key | Elsevier ,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,Searching for information using AI.,A search platform that clinicians can use when searching for current evidence- based medical practices. ,Summary of medical literature or answers to medical questions based on a search prompt.,Initiated,Both,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, ACS NSQIP Risk Score,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,Clinical decision support by providing surgical risk assessment,Provides a surgical risk assessment for a patient based on a set of values input by a provider.,Operation and Maintenance,Both,12/16/2013,11/1/2014,11/1/2014,,,"none, free product",Yes,Veterans Health Administration,,Yes,There is not identifiable information input into the system by end users using the tool as it does not collect the HIPAA safe harbor elements.,No,No,No,,"The history of the ACS/NSQIP initially started in response to issues around surgical quality and the VA in the VASQIP national database. There are around 600 or so publications regarding the tool including a number with use case assessment using VA data. There may be agency owned data used previously in the ACS/NSQIP development process, but I cannot confirm that based on review work completed so far, but given the history of association with VASQIP and the VA, the model was in part influenced originally from VA findings per the history on the tool. The history of the tool's development is noted here: https://www.facs.org/quality-programs/data-and-registries/acs-nsqip/history/","Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.","[""Sex/Gender"",""Age"",""Ability Status""]",,No,No – agency does not have access to source code.,There are a number of links to information on the tool noted here. https://riskcalculator.facs.org/RiskCalculator/about.html,No,,,No,,No,,Other,not applicable,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Avicenna LVO,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,"The function of the AI is to detect large vessel occlusion in the anterior intracranial circulation (distal Internal Carotid Artery, M1 and proximal M2 Middle Cerebral Artery) on Head CT Angiography exams to aid and expedite diagnosis.",The input is DICOM Head CT image data from VHA facilities that submit the exam to VHA NTP for interpretation. The output is DICOM object written to the NTP Picture Archive Communication System (PACS) for review by the interpreting VHA NTP Diagnostic Radiologist. Future output will also include a DICOM structured report to the VHA NTP PACS Powerscribe dictation software.,Implementation and Assessment,Both,3/1/2021,6/1/2022,12/1/2024,,Developed with contracting resources.,36C10B23C0025,Yes,Veterans Health Administration,,No,N/A,Yes,Yes,Yes,,"N/A - FDA 510K cleared COTS product, trained using OEM data","Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.","[""Age""]",,No,No – agency does not have access to source code.,N/A,Yes,National Teleradiology Program Next Generation Picture Archive and Communication System #2418,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Ysio Max,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf13/K133259.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Transpara Breast Care,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,"The function of the AI is to analyze digital mammograms to identify and characterize patterns of breast tissue suspicious for breast cancer, potentially identifying cancers more quickly and at an earlier stage.",The input is DICOM mammography image data from VHA facilities that submit mammography exams to VHA National Teleradiology Program (NTP) for interpretation. The output is DICOM object written to the NTP PACS for review by an interpreting VHA NTP Diagnostic Radiologist. Future output will also include a DICOM structured report to the VHA NTP PACS Powerscribe dictation software.,Implementation and Assessment,Both,6/1/2022,6/1/2022,12/1/2024,,Developed with contracting resources.,36C10B23C0025,Yes,Veterans Health Administration,,No,N/A,Yes,Yes,Yes,,"N/A - FDA 510K cleared COTS product, trained using OEM data","Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.","[""Age"",""Sex/Gender""]",,No,No – agency does not have access to source code.,N/A,Yes,National Teleradiology Program Next Generation Picture Archive and Communication System #2418,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, "XIDF-AWS801, Angio Workstation (Alphenix Workstation), V9.5",Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf23/K232526.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, BlackBox Code research,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Mission-Enabling (internal agency support),,Creating visual representations of data sets for reports and presentations using AI.,"We do a lot of coding in various languages java, SQL, Mquery, DAX, Fx, python and R. While being able to use this product and other products like ASkSage and ChatGPT. It has had a lot of use in identifying code errors, improving performance of code to better load data in other systems in VA and providing insights on new ways to code in many programs. ","The AI system's outputs can be categorized into several types, depending on the task or question being addressed. Here are some examples: Math and Logic Problems Step-by-step solutions to math problems, including algebra, geometry, calculus, and other branches of mathematics Logical explanations and derivations for logic problems, including propositional and predicate logic Coding and Programming Code snippets in various programming languages, such as Python, Java, C++, and JavaScript, with clear explanations and comments Detailed code reviews and suggestions for improvement Algorithmic solutions to programming challenges and problems Creative Writing and Role-Play Original creative writing pieces, such as short stories, poems, or dialogues, in response to prompts or requests Engaging role-play scenarios, including character development and interactive storytelling Analysis and Discussion In-depth analysis of topics, including scientific concepts, historical events, and cultural phenomena Well-structured and coherent arguments, with evidence and supporting examples Engaging discussions and debates on various topics, with respectful and open-minded dialogue Teaching and Explanation Clear and concise explanations of complex concepts, broken down into manageable parts Step-by-step tutorials and guides for learning new skills or subjects Patient and supportive feedback, with suggestions for improvement and further learning General Discussion and Q&A Concise and accurate answers to factual questions, with sources and references when available Engaging and respectful conversations on various topics, with active listening and thoughtful responses Overall, the AI system's outputs are designed to be informative, engaging, and helpful, with a focus on clarity, accuracy, and respect for the user's time and interests",Initiated,Neither,8/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Synthetic Data Creation,Department of Veterans Affairs,VA,VBA: Veterans Benefits Administration,Education & Workforce,,None of the above.,"Synthetic (non-PII) test data creation, using Generative AI, at the scale and representation of the diverse characteristics of the beneficiary population necessary to support EDU UAT.  ",Synthetic beneficiary population data,Initiated,Neither,7/18/2024,,,,Developed with contracting resources.,VA118-16-D-1013 36C10D21N0007,No,,,No,,No,Yes,No,,GI Bill beneficiary demographic data. This model's purpose is to replicate real data but with synthetic identifiers to avoid using PII in User Acceptance Testing.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.","[""Race/Ethnicity"",""Sex/Gender"",""Age"",""Other""]","This model is currently in the exploratory phase, so these demographic factors are not exhaustive as of yet.",No,No – agency does not have access to source code.,,Yes,Digital GI Bill,Less than 6 months,No,,No,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Billie GPT,Department of Veterans Affairs,VA,VBA: Veterans Benefits Administration,Education & Workforce,,None of the above.,Generative AI knowledge portal to support School Certifying Officials in addressing questions regarding VA education benefits and use of the Enrollment Manager System.,Generative AI response to the user query,Acquisition and/or Development,Neither,3/5/2024,4/29/2024,,,Developed with both contracting and in-house resources.,VA118-16-D-1013 36C10D21N0007,No,,,Yes,"Rigorous review of training data, inputs, and regular UAT testing to ensure accuracy.",No,Yes,No,,"Training: VA-created user guides, GI Bill-related documents, and training resources. Evaluating performance: Call Center volume, User feedback scores, Increased use over current rule-based NLP bot.","Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",,,No,No – agency does not have access to source code.,,Yes,Digital GI Bill,Less than 6 months,No,,No,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, X100HT with Slide Loader with Full Field Peripheral Blood Smear (PBS) Application,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf22/K220013.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, WRDensity by Whiterabbit.ai,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf20/K202013.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Workflow Box,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf18/K181572.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, BTSSS 3542 Review,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Government Services (includes Benefits and Service Delivery),,Inputting large amounts of data from paper forms into a digital system using AI.,"The intended purpose of the AI model is to streamline the claims processing workflow by automatically extracting relevant information from patient-filled paper forms and organizing the corresponding scans chronologically. This automation eliminates the tedious task of manually renaming and sorting files, significantly reducing the time staff spend on these activities. The expected benefits include a marked decrease in manual labor hours, leading to lower operational costs and minimizing the need for overtime. Ultimately, this efficiency allows staff to focus more on patient care and other critical tasks, improving overall workflow productivity.","The AI model outputs a renamed version of the original file, moving it to a designated SharePoint folder with the relevant information extracted from the patient form. Additionally, specific columns in the file are updated with data to facilitate easy sorting by dates. This streamlined process ensures quick access to organized claims information and enhances overall efficiency.",Implementation and Assessment,Neither,,,,,,,,,,,,,,,,,,,,,,,Yes,BTSSS ID: 713,,,,,,,,,,,,,,,,,,,,,,,,,,, Withings Scan Monitor 2.0,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf23/K230812.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, HTM-LLM,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Mission-Enabling (internal agency support),,None of the above.,RAG chatbot using technical service manuals to aid Healthcare Technology Management staff in medical device maintenance and troubleshooting,Chatbot responses,Initiated,Neither,5/1/2024,,,,Developed in-house.,,No,,,No,n/a,No,No,No,,"none, uses manufacturer service manuals","Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,Yes – source code is publicly available.,https://github.com/ahncs/AIE3/tree/main/Final%20Project,No,,,No,,No,,Other,still haven't provisioned VA resources yet.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Clinician-Administered PTSD Scale for DSM-5 (CAPS-5) Clinician Training Simulator,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Education & Workforce,,None of the above.,"The CAPS-5 Clinician Training Simulator comprises three online courses, available in TMS and TRAIN, that provide instruction in the administration and scoring of the CAPS-5. The courses use voice recognition, 3-D technology, and artificial intelligence to create an immersive learning experience. The courses are intended for clinicians, trainees, researchers, and examiners in VA and in the community. They use AI in the following ways: 1. In the three current CAPS-5 virtual patient courses, learners have the option to speak the prompts, with their spoken prompts transformed to text using speech recognition. 2. The learners’ prompts, whether spoken or typed, are programmatically compared to the required prompts in the CAPS-5. The system analyzes whether the prompts were delivered exactly (“verbatim”), with slight variations (“paraphrased”), or with unacceptable variations (“off-script”). 3. One of the courses analyzes learner prompts in order to trigger comments –corrections or praise -- from a virtual coach. 4. We’re currently revising one of the courses to use generative AI. Here’s how the vendor described it in their response to our solicitation: “This new design will also leverage a separate backend to handle AI multi-agency and retrieval augmented generation (RAG) to ensure large language models (LLMs) used will always operate as designed.” This will be a closed system that does not pull information from or input information to publicly-available platforms (e.g., GPT-4, Bard). More information about the three courses and access to them is available at https://www.ptsd.va.gov/professional/continuing_ed/caps5_clinician_training.asp. ","The AI system outputs, narrowly defined, are the virtual patients' responses to the learners' prompts. More broadly, the system is designed to help learners learn to more accurately administer and score the CAPS-5. Learners receive a summary document describing their performance at the conclusion of each course.",Operation and Maintenance,Neither,10/15/2018,11/29/2018,3/13/2020,,Developed with contracting resources.,3610G8Q9155,No,,,No,,No,No,No,,n/a,,,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,,No,,No,,No,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, WellDoc BlueStar,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf19/K190013.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Electronic Virtual Assistant (e-VA),Department of Veterans Affairs,VA,VBA: Veterans Benefits Administration,Government Services (includes Benefits and Service Delivery),,None of the above.,"The electronic Virtual Assistant (e-VA) information system is a product acquired from SaraWorks (formerly The Career Index (TCI)) by the Veterans Benefits Administration (VBA) Veteran Readiness & Employment (VR&E) Business Line. e-VA is a breakthrough digital assistant that does most of the work a human assistant can do in terms of client follow-up, data entry, and documentation. The primary objective for implementing e-VA is to provide virtual support services that will alleviate administrative tasks from Vocational Rehabilitation Counselors (VRCs), which enable them to focus their time on the Veterans’ needs, fulfilling VR&E’s mission of guiding Veterans to successful outcomes. e-VA is the only digital assistant focused on applying these autonomous capabilities to the complex field of human services. The e-VA system will create reminders for appointments, grades, receipts, etc. and accept documents and images, in addition to connecting program participants directly to their Counselor. The application also enables automated routine electronic communication with program participants through either text message and/or e-mail. Participants can interact with the e-VA application via text or email to schedule and reschedule appointments, respond to reminders, and submit supporting documents and receipts related to their VR&E case. The e-VA information system is single-tenant, which resides in SaraWorks private AWS GovCloud. It is not a public-facing system. The SARA Sync windows executable will receive information from the e-VA system, such as supporting documents, receipts, etc., case notes, or updating of status on their case. This information are transmitted to the VA database of record for storage.","Allow VRCs to focus on providing world-class service to our Veterans. The e-VA system will create reminders for appointments, grades, receipts, etc. and accept documents and images, in addition to connecting program participants directly to their Counselor. The Case Manager ID will be used to identify case managers and assigned caseload which will determine the location of the requested appointment. The application will also enable automated routine electronic communication with program participant through either text message and/or e-mail. Participants can interact with the e-VA application via text or email to schedule and reschedule appointments, respond to reminders, and submit supporting documents and receipts related to their VR&E case. The SARA Sync windows will receive information from the e-VA system, such as supporting documents, receipts, etc., case notes, or updating of status on their case. This information are transmitted to the VA database of record for storage.",Operation and Maintenance,Both,6/1/2019,6/1/2019,6/20/2020,,Developed with both contracting and in-house resources.,36C10E19P0165,No,,,Yes,N/A,Yes,Yes,No,,We do not train e-VA,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.","[""Other""]",None,Yes,No – agency does not have access to source code.,N/A,Yes,electronic Virtual Assistant (e-VA) VASI: 2419,,No,,Other,e-VA does not do predictive modeling,Other,e-VA does not do predictive modeling,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, WAVE Clinical Platform,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf17/K171056.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, "VX1, VX1+",Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf22/K223516.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Sybil - Lung Cancer Prediction Model by MIT,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,"To be used to personalize screening regime, calling high risk patients earlier to catch cancers in the early stages and potentially reducing screening burden for low risk patients.","It takes as input a lung low-dose CT scan, and produces a JSON return object with predictions of the odds of developing cancer within the next 6 years (one prediction for each year).",Acquisition and/or Development,Both,4/10/2024,10/16/2023,,,Developed in-house.,,Yes,Veterans Health Administration,,No,,Yes,No,No,,We (Dayton HTM) have developed a web based front end to point DICOM images to the API. The resulting JSON cancer prediction results are transcribed into a excel table for validation. Our aim is to have the model on server behind the VA's isolation architecture (MDIA or SDAI) and have results stored in SQL database. With access secured via LDAP.,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ","[""Other""]",Low Dose CT Images of lung are exclusively used.,Yes,Yes – source code is publicly available.,https://github.com/reginabarzilaygroup/Sybil,No,Sybil,6-12 months,Other,The system/model is isolated to one air-gapped workstation. Our aim is to have the system approved via the Enterprise Risk Assessment process. I will then request an isolated server to house the AI model.,Yes,,Other,The provisioning request via the EWR has not been started yet. The system as it stands now is still air-gapped.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, eCaremanager (Philips),Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,Creating visual representations of data sets for reports and presentations using AI.,"eCaremanager integrates clinical, physiological, and demographic data from all patients entered into the National TeleCC Program. The data are presented graphically to users. Additionally, the data are used to generate acuity scores to assess each patients acuity in comparison to all other patients. Trend and threshold alerts are also generated to notify providers to take a closer look at those patients and initiate treatments to remediate derangements and facilitate return to homeostasis and health.",The outputs of eCaremanager are acuity scores and trend and threshold alerts. The software also integrates and presents patient care information to enable providers to care for larger number of patients and provide critical care at a population level.,Operation and Maintenance,Both,1/1/2010,1/1/2010,1/1/2010,,,,,,,,,,,,,,,,,,,,Yes,TCC ID: 2209,,,,,,,,,,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, VUNO Med-DeepBrain,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf23/K231398.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Policy AId,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Mission-Enabling (internal agency support),,Searching for information using AI.,"Policy lookup is a time-consuming task in the VA. This challenge was identified as an ideal candidate for generative AI product development. After completing AI engineering and product development training, our internal team developed a Retrieval-Augmented Generation (RAG) application that accepts policy-related questions and provides relevant answers based on Medical Center reference materials. Expected Benefits: Time spent on finding/referencing facility policy decreases, increasing efficiency and employee satisfaction. Standardization of meeting minutes per Dayton governance structure. ",LLM produces feedback in text with links to the associated medical center publications.,Implementation and Assessment,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, "Voluson Expert 22, Voluson Expert 20, Voluson Expert 18",Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf23/K231965.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Volta AF-Xplorer,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf23/K232616.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Biotronik Home Monitoring System,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,"The Biotronik home monitoring system collects data transmitted by patients with implanted cardiac monitors from their homes for review by clinicians on a password protected Web page. Transmissions are made by the monitors when certain criteria are met suggesting an important heart rhythm abnormality is present. A proprietary industry-created AI system evaluates the transmitted data and classifies whether the findings that triggered the transmissions are true arrhythmias or false positives. The home monitoring system for BioMonitor IV implanted cardiac monitor uses AI to detect false positives for atrial fibrillation, bradycardia, tachycardia and pauses. These technologies leverage machine learning algorithms and vast datasets to provide more accurate diagnoses and patient care. The staff of the VA National Cardiac Device Surveillance program and the patients’ providers at local VA clinics use this AI when evaluating patient implanted cardiac monitor transmissions.",It filters possible arrhythmia events in to those that are false positive and those that are true positive.,Initiated,Both,4/10/2024,,,,,NA,Yes,Veterans Health Administration,,No,This a commercial product approved by the FDA,No,No,Other,This is a commercial product. We are not involved in development.,This is a commercial product. We are not involved in development.,,,This is a commercial product. We are not involved in development.,No,No – agency does not have access to source code.,not open source,No,,,,,,,,,,,,,,,,,,,,,,,,,,,, "Synthetic Data Generation: Experimenting with OpenSource, Third Party and GenAI ",Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Other,Research: creation of synthetic data for model training and development,None of the above.,Improve agility of research and operational dev,Synthetic patient clinical and demographic data,Initiated,Neither,3/1/2024,,,,Developed in-house.,,No,,,No,n/a,No,No,Yes,,CDW-ng is going to be used to train,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.","[""Other""]",all of the above but none is saved. only the synthetic data is saved.,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,2266,More than 12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, CareLink Home Monitoring,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,The CareLink home monitoring system collects data transmitted by patients with implanted cardiac monitors from their homes for review by clinicians on a password protected Web page. Transmissions are made by the monitors when certain criteria are met suggesting an important heart rhythm abnormality is present. A proprietary industry-created AI system evaluates the transmitted data and classifies whether the findings that triggered the transmissions are true arrhythmias or false positives. The home monitoring system for Carelink Reveal Linq and Linq II uses AI deep learning algorithms flowing into the CareLink network to remove false AFib and false pause episodes. The staff of the VA National Cardiac Device Surveillance program and the patients’ providers at local VA clinics use this AI when evaluating patient implanted cardiac monitor transmissions.,It eliminates false positive atrial fibrillation and pause events recorded by Medtronic implanted loop recorders when you are reviewing data from these devices on the Medtronic Carelink web page.,Initiated,Both,11/13/2023,,,,,NA,Yes,Veterans Health Administration,,No,This is an FDA approved use of AI,No,No,Other,This is an commercially developed and deployed FDA approved AI use. The VA was not part of its development.,This is an commercially developed and deployed FDA approved AI use. The VA was not part of its development.,,,,No,No – agency does not have access to source code.,Not open source,No,,,,,Other,NA,Other,NA,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Nuance Dragon Medical One (DMO),Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Mission-Enabling (internal agency support),,Inputting large amounts of data from paper forms into a digital system using AI.,Documentation is entered into the electronic health record leveraging voice to text technology,Voice-generated content and turn it into clinical documentation,Operation and Maintenance,Both,,,,,,,,,,,,,,,,,,,,,,,Yes,EWSR DMO -E ID: 1206,,,,,,,,,,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Stratification Tool for Opioid Risk Mitigation (STORM) predictive model ,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,"The STORM model is a predictive model to estimate risk of a suicide or overdose-related health care event or death in the next year. The model uses patient level predictors extracted from the patient’s medical records (into the VA’s Corporate Data Warehouse), such as diagnoses, prescriptions, and health care utilization, etc. This predictive model is used for three purposes. (1) Primary use case is to identify patients at elevated risk of suicide or overdose-related events for referral to an interdisciplinary team for clinical case review. This team makes treatment recommendations to share with treating providers and documents them in the medical record. The STORM algorithm is used to identify patients to refer to this program at each facility. Assignment to this case management program has been shown to reduce all-cause mortality in patients estimated at “very high” risk on the STORM predictive model by 22% in the next 4 months. (2) Secondary use case is to help risk stratify patients based on prior health care interactions to support augmentation of risk mitigation interventions and treatments for patients with patterns of care associated with risk of suicide or overdose events. Risk estimates are provided as a sorting/filtering and informational feature on the STORM clinical decision support system. This decision support system enables population management of patients prescribed opioids or with opioid use disorders, as well as providing curated information to support treatment planning for individual patients. (3) The last use case is to meet the requirements of the Comprehensive Addiction Recovery Act (CARA) mandating that clinicians use VA decision support tools to review patient risks prior to initiating opioid therapy for pain. Direct synchronized links from the VISTA medical record to the STORM report integrate this decision support summary information into clinician workflows as a support for risk assessment before writing and initial opioid prescription. Each of these use cases is built into existing clinical workflows completed by overlapping, but different health care providers. For use case 1, designated interdisciplinary risk review teams at each health care system use the risk estimates to determine which patient require risk review. For use case 2, VA health care providers (e.g., opioid prescribers) and teams (e.g., primary care and general mental health teams) use the risk estimates in combination with decision support recommendations of interventions to consider facilitate and augment risk assessment and treatment planning for their patients on opioid analgesics or being treated for opioid use disorder. For use case 3, VA medication prescribers or their delegated team members use the categorized risk scores in the context of curated descriptive data from the patient’s medical record to review risk factors and consider risk and benefits of treatment options for pain management before prescribing opioids. Risk considerations are documented in a clinical note for tracking. Use of STORM is expected to improve decision-making about opioid prescribing and treatment of patients with opioid use disorders, as well as consistency of use of clinical practice guideline recommended treatment practices for safe management of patients exposed to opioids. STORM also focuses on ensuring recognition of patient risks and facilitating care coordination across treatment providers for complex patients with multiple interacting disorders. By improving use of risk mitigation practices and treatment interventions, we expect STORM will reduce adverse outcomes for patients exposed to opioids. Program evaluations support this expectation, finding that implementation of the targeted interdisciplinary review requirements associated with use case 1 significantly reduced all cause mortality in the targeted patient population. Links: https://psycnet.apa.org/buy/2017-03732-004 https://www.tandfonline.com/doi/10.1080/08897077.2018.1540376 https://link.springer.com/article/10.1007/s11606-022-07622-1 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10698768/","The STORM predictive model estimates the risk of having an overdose or suicide-related event in next year for all patients currently receiving a VA opioid analgesic prescription with nightly updates. For all other active VHA patients, the STORM predictive model estimates the hypothetical risk of having an overdose or suicide-related event in next year, if the patient was prescribed or otherwise had access to low, medium, or high dose opioid analgesics. These risk estimates are provided to clinicians on a family of dashboards, both as percentage likelihood of an event in the next year, and as a categorical grouping of estimated low, medium, high or very high risk. Paired with this risk estimate is a summary of all risk factors identified and considered in generating the risk estimate for that patient, and recommendations for risk mitigation and treatment options to consider. These recommendations are tailored based on patient characteristics identified in the medical record (e.g. Medications for opioid use disorder are recommended for patients seen for an opioid use disorder diagnosis) and patient receipt of these interventions is tracked and displayed on the dashboard. The STORM dashboards are used by clinicians as an informational tool to summarize and organize medical record information to support risk reviews during treatment planning (e.g. when considering an opioid prescription), for management of patients prescribed opioids or treated for opioid use disorder, and for interdisciplinary case reviews of very high risk patients to ensure safe and effective care coordination and treatment planning across health care providers.",Operation and Maintenance,Both,6/1/2012,6/1/2012,6/1/2015,,Developed in-house.,,No,,,No,NA,Yes,Yes,Yes,,"Clinical data came from the VHA National Corporate Data Warehouse, an archival data set that contains comprehensive patient-by-patient, encounter-by-encounter clinical and administrative data derived from the Department of Veterans Affairs’ electronic medical records. Data regarding vital status and cause of death were from the National Death Index, a centralized database maintained by the National Center for Health Statistics of death record information on file in state vital statistics offices.","Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.","[""Marital Status"",""Sex/Gender"",""Age""]",,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,VA Corporate Data Warehouse - 1152,Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,Yes,"Impact evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution and has utilized randomized experiments with a control group or other counterfactual, social systems analysis, or other rigorous research methodologies to evaluate impact and identify potential harm to users as well as broader groups of people.","(1) Providers might over-rely on statistical risk assessment. This risk was a concern based on known tendencies of some providers to over-interpret recommendations as requirements. This tendency was observed in a minority of providers and addressed explicitly in training and training materials moving forward. (2) The program could be clinically inefficient and use provider time that might be more effectively focused elsewhere. This is a concern for all VHA clinical programs, and VHA management consistently seeks to optimize clinical time and attention towards achieving the greatest patient benefit. (3) The program could lead to unfair allocation of clinical resources if the model is biased and other clinical programming does not compensate for areas of model under-performance. Here, a targeted case review is required for patients identified as ""very high"" risk per the STORM predictive model, though all clinical services are available and recommended for all patients based on clinical indication in the STORM decision support system. After review of the common risks template, the following risks were identified: Fair & Equitable (FE) - Algorithm target not reflective of real-world outcome of interest Response: There is a risk that changes in population exposures and behaviors related to opioids may alter patterns of risk away from those the model was trained on. Transparent & Explainable (TE) - Degradation of End-User Trust and Ineffective Challenge or Remedy Processes Response: This is a risk, there has been public misunderstanding of how VA predictive models are developed and used. VA has responded and clarified misconceptions where concerns have been raised. Accountable & Monitored (AM) - Performance Efficacy and Fairness (e.g. Model/Data Drift, Model Degradation, or inappropriate application) Response: This is a risk, model performance is degrading over time and an update is in process. However all identified patients are clinically complex and appropriate for clinical attention. Accountable & Monitored (AM) - User-Introduced Errors Response: There is a low risk, there is a community of practice and regular monthly education to support proper implementation of the VA opioid safety program, including the STORM decision support system and predictive model. Accountable & Monitored (AM) - System Performance Not as Intended Response: There is a low risk, team has validation processes to review all steps.",TRUE,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,Online - in the terms or instructions for the service.,,Yes,"VA has examined performance of the STORM model for estimating overdose and suicide -related risk among Veterans in VHA care, stratified by patient sex, race and ethnicity. The first iteration of model performance evaluation is described in List et al., 2023, available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10698768/. The main finding from this evaluation was that the model tended to underestimate risk in non-white populations in years subsequent to the year the model was trained on. This underestimation tracked population level changes in opioid overdose rates across this period, as there was a significant increase in observed overdose rates in non-white populations between the model training year and later years. This finding emphasized the importance of regular updating of the predictive model to address changes in risk contributions in the U.S. population over time. Additional efforts to optimize bias evaluation methods were undertaken utilizing the STORM model as an example (Meerwijk et al., 2024). While this evaluation emphasized the complexity of bias evaluation, it also suggested opportunities for improvement of model performance, particularly for older adults, who tend to utilize VHA mental health services less than younger VHA patients. Guided by these findings, VHA is actively working on an update to the STORM predictive model, incorporating a significantly expanded set of candidate predictor variables in hopes of better capturing indications of risk across subpopulations and optimizing the model to reflect evolving trends in opioid exposure and outcomes. As part of this effort, VHA has built a decision support infrastructure to enable remodeling, model evaluation, and model deployment into VHA clinical decision support systems on an annual or as-needed cycle. References: Meerwijk EL, McElfresh DC, Martins S, Tamang SR. Evaluating accuracy and fairness of clinical decision support algorithms when health care resources are limited. J Biomed Inform. 2024 Aug;156:104664. doi: 10.1016/j.jbi.2024.104664. Epub 2024 Jun 6. PMID: 38851413. List JM, Palevsky P, Tamang S, Crowley S, Au D, Yarbrough WC, Navathe AS, Kreisler C, Parikh RB, Wang-Rodriguez J, Klutts JS, Conlin P, Pogach L, Meerwijk E, Moy E. Eliminating Algorithmic Racial Bias in Clinical Decision Support Algorithms: Use Cases from the Veterans Health Administration. Health Equity. 2023 Nov 30;7(1):809-816. doi: 10.1089/heq.2023.0037. PMID: 38076213; PMCID: PMC10698768.","[""Direct user testing""]",,No – it is not operationally practical to offer this.,"While we do have a process via which a patient can request to not have risk estimates created, the output of the system only consists of information to guide clinical decision-making. All decisions and outcomes are driven by the clinician treating the patient.",Yes, Vitrea CT Brain Perfusion,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf18/K181247.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Deep-Learning approaches to develop candidate lists of terms for use in text searches ,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,"The intended use is to increase the comprehensiveness of term lists to be used in text-matching approaches to identifying concepts of interest in free-text medical record notes. Text-matching approaches are an efficient method for identifying text of interest in medical notes. The success of text-matching approaches depends on the quality of term lists used to search for a concept of interest. While humans can generate word lists for text-matching approaches, the generated word lists are likely to be limited and biased by the vocabulary, language habits, population exposure, and local dialects of the human generating the list. Deep learning methods can help to augment candidate term lists to help overcome some of these challenges. Once generated these augmented candidate terms are reviewed by subject matter experts for appropriateness and accuracy of the suggested terms for the concept and use case of interest. Here we (1) query pre-trained large language models (e.g., ChatGPT or Llama) through a user interface that allows prompting (such as the OpenAI API or CodeLlama) and/or (2) query word and phrase “embedding” models (e.g., Word2Vec and Phrase2Vec) to expand candidate term lists for review by subject matter experts. Our team has found both approaches effective for expediting the generation of more comprehensive term to concept mappings for use in text-matching algorithms that increase the accuracy of clinical concept extraction. We expect this use of AI to improve initial brainstorming of terms related to a given concept, resulting in a more sensitive approach to finding information of interest in free-text clinical notes. ","The outputs of this use of AI is a candidate list of terms related to a concept of interest. This candidate list is combined with suggestions generated by human experts and then these are reviewed by a subject matter expert to generate a final list of terms for use in text-matching algorithms to extract medical record free text mentions of interest. For the pre-trained Large Language model approach, the input is a query prompting for synonyms of terms related to the concept of interest. The output is the answer provided by the LLM, including the suggested candidate terms and the original terms that were stated in the query. For the word and phrase embedding models, the input is an initial list of terms that are relevant to the concept of interest and the output are the suggested candidate terms, ranked by their statistical similarity to the original terms, based on the transformation of a clinical corpus into vector space or an “embedding model”. Regardless of the type of model used, all suggested candidate terms are reviewed by subject matter experts to ensure quality term lists are used in all text searches.",Operation and Maintenance,Neither,10/1/2022,10/1/2022,10/1/2022,,Developed in-house.,,No,,,No,NA,No,No,Other,The subject matter expert teams relevant to the concept of interest create the needed queries and initial term lists.,NA,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Yes,,Yes,,Yes,,Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Vereos PET/CT,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf21/K211764.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Recovery Engagement and Coordination for Health – Veterans Enhanced Treatment (REACH VET) 1.0 predictive model,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,"The REACH VET 1.0 model is a predictive model using predictors extracted from the patient’s medical records (into the VA’s Corporate Data Warehouse), such as diagnoses, prescriptions, and health care utilization, to estimate statistical risk of a suicide death in the next month. This predictive model is used for two purposes. (1) Primary use case is to support identification of patients at elevated statistical risk of suicide in the next month for referral to a clinical case review and outreach program. Staffing guidance to ensure each health care system is staffed to conduct the case review and outreach protocol is established per clinical policy. To date, the population recommended to the clinical case review program has been limited to a number equivalent to 0.1% of the total treated patient population at each facility each month; the number of patients in the program may be modified based on guidance from the Office of Suicide Prevention. The REACH VET 1.0 statistical risk estimates contribute to an algorithm used to identify patients to refer to this program at each facility. (2) Secondary use case is to help risk stratify patients based on prior health care interactions to support augmentation of clinical attention and interventions for patients with patterns of care associated with high suicide risk. This is used, for example, to support clinical review of relevant health care history during triage of Veteran Crisis Line calls and during risk assessment of patients in mental health care. For use case 1, REACH VET coordinators and REACH VET providers use the list and supporting information to guide execution of the targeted prevention program. For use case 2, Veterans Crisis Line responders and VHA health care providers use the categorized risk scores in the context of curated descriptive data from the patient’s medical record to review risk factors towards optimizing assessment and clinical follow-up. The REACH VET 1.0 model is expected to identify a population of patients at elevated risk of suicide and other negative outcomes for review by clinicians; clinicians conduct telephone outreach to identified patients when they deem such outreach appropriate. This model and targeted prevention in which it is used (i.e. the REACH VET program) is used to augment VHA's extensive clinical suicide prevention program. While VHA conducts universal screening for suicide risk by asking patients structured questions regarding suicidality, this screening process does not identify all patients who go on to die of suicide. The REACH VET model is intended to identify patients who might be missed in clinical assessment or whose risk may have changed since their last assessment. Here, the REACH VET model identifies patients at statistical risk of suicide, that is they share similarities in their health care data to patients who did die of suicide in the past. The patients at highest statistical risk are highlighted to mental health providers for additional review. This targeted prevention program has been nationally implemented in VHA health care systems since 2017 and an evaluation of the effects of the REACHVET program was conducted. This evaluation found that REACH VET implementation was associated with greater treatment engagement and new safety plan documentation and fewer mental health admissions, emergency department visits, and suicide attempts. Full findings of this program evaluation are available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8524305/","The REACH VET predictive model estimates the likelihood of dying of suicide in the next month for each active patient in VHA. This estimate is used as part of an algorithm that considers factors such as the size of the health care system, current patient treatment engagement, and whether the patient has had previous engagement with the REACH VET program to generate a dashboard with patients at high statistical risk of suicide for review by a REACH VET coordinator and identified clinicians at each health care system. The REACH VET dashboard provides information about identified risk factors for the included patients and supports a coordinated case review and, where applicable, clinical outreach process to support identification of care gaps and offer additional services as appropriate.",Operation and Maintenance,Both,1/1/2014,1/1/2014,11/1/2016,,Developed in-house.,,No,,,No,NA,Yes,Yes,Yes,,"As quoted in publication 'Clinical data came from the VHA National Patient Care Database, an archival data set that contains comprehensive patient-by-patient, encounter-by-encounter clinical and administrative data derived from the Department of Veterans Affairs’ electronic medical records. Data regarding vital status and cause of death were from the National Death Index, a centralized database maintained by the National Center for Health Statistics of death record information on file in state vital statistics offices. We conducted searches with established procedures.24 We constructed 3 samples: (1) model development, (2) model validation, and (3) prediction. The development and validation samples contained data for patient-months from all patients who died from suicide from October 1, 2008, to September 30, 2011, and had received VHA services in the previous 2 years (case patients) and a random 1% of patients who survived the month and received VHA services in the previous 2 years (control patients). We randomly assigned half of the case patients and half of the control patients to the development sample and half to the validation sample. Each contained patient-months from 3180 case patients and 1 056 004 control patients.' (McCarthy et al, 2015). Data from the VA Corporate Data Warehouse was used to evaluate performance of the models.","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.","[""Age"",""Sex/Gender"",""Marital Status"",""Race/Ethnicity"",""Ability Status""]","Note that ""Ability status"" is service connected disability status in this model.",Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Corporate Data Warehouse - 1152,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,Yes,"Impact evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution and has utilized randomized experiments with a control group or other counterfactual, social systems analysis, or other rigorous research methodologies to evaluate impact and identify potential harm to users as well as broader groups of people.","Our main concerns regarding risks are as follows: (1) Providers might over-rely on statistical risk estimate. To reduce this risk, we provide information regarding which risk factors were identified by the predictive model to encourage transparent understanding of the basis for the model estimate. Moreover, the clinical prevention program provides only general guidelines to clinicians indicating that identified patients should be reviewed. This forces clinicians to rely on their clinical skills to plan intervention responses. (2) The program could be clinically inefficient and use provider time that might be more effectively focused elsewhere. This is a real risk which may evolve over time as other components of the overall VA suicide prevention program improve, and potentially reduce the need for this targeted prevention program. VHA conducts ongoing evaluations to assess for potential inefficiencies in the model and test methods to improve clinical efficiency. For example, VA is currently piloting a process to expand focus of the REACH VET program to patients not currently engaged in mental health care. (3) The program could lead to unfair allocation of clinical resources if the model is biased and other clinical programming does not compensate for areas of model underperformance. This is also a risk and thus VHA evaluates model bias across key demographic populations and actively develops methods to reduce model bias or address bias by adjusting other components of the treatment system to compensate. After review of the common risks template, the following additional risks were identified: Fair & Equitable (FE) - Algorithm target not reflective of real-world outcome of interest Response: There is a minor risk that the county coroners who assign cause of death might have miscategorized some Veteran deaths, reducing the validity of the outcome variable used. Transparent & Explainable (TE) - Degradation of End-User Trust and Ineffective Challenge or Remedy Processes Response: This is a risk. There has been public misunderstanding of how the model performs and is used. VA has responded and clarified misconceptions in the public forums in which misunderstanding has arisen. Accountable & Monitored (AM) - Performance Efficacy and Fairness (e.g. Model/Data Drift, Model Degradation, or inappropriate application) Response: This is a risk, model performance is degrading over time and an update is in process. However all identified patients are clinically complex and appropriate for clinical attention. Accountable & Monitored (AM) - User-Introduced Errors Response: There is a low risk, there is a community of practice to support proper implementation of the REACH VET Program, which is supervised and trained by national program leads. Accountable & Monitored (AM) - System Performance Not as Intended Response: There is a low risk, the internal VA team that manages the model and risk estimates based on it has validation processes in place to review all steps of the process.",TRUE,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,Telephone,,Yes,"VA has examined suicide risk concentration among Veterans in VHA care, stratified by patient sex, race and ethnicity. Model performance was reasonably similar across demographic subgroups. Because the REACH VET clinical program is just one augmentative component of the overall suicide prevention program, weaknesses in the ability of the REACH VET algorithm to identify high risk patients for intervention have been considered in design of other components of the overall suicide prevention program and mitigated where possible. For example, the REACH VET model lacks information about patients who have had minimal engagement with the health care system for suicide risk factors they may have experienced. As such, the model will inherently underestimate risk for new patients and patients experiencing new episodes of symptoms or stressors that could elevate risk. To combat this weakness, VA implemented a universal suicide risk screening process, including at least annual screening of all patients in primary care and mandatory suicide screening in VA emergency departments which are likely to be the entry point for patients with new challenges that are not yet documented in medical records. This screening triggers mandatory follow-up for positive cases by a clinician with mental health care training, who can offer services (as would be done in REACH VET triggered outreach). As another example, some known risk factors for suicide were not well captured in medical records at the time of model training. VHA has implemented clinical screening programs for some of these risk factors (e.g. military sexual trauma, food insecurity) and trained clinicians regarding their contribution to risk and the importance of offering services to address challenges. Likewise, VA decision support systems that provide drill-in summaries of suicide related risks for REACH VET and other VHA patients (i.e. CRISTAL) summarize and display information from these clinical screeners with other information about suicide risk factors to facilitate clinical evaluation and response to suicide risks. While it is impossible to build a REACH VET model that is not affected by biases in detection of risk factors due to variation in access to/utilization of VA health care, VHA has taken steps to address expected/known gaps in model performance through implementation of other suicide prevention programming.","[""Post-transaction customer feedback collections"",""Other"",""Public hearings or meetings""]",,Yes,,Yes, Venue; Venue Fit; Venue Go,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfPMN/pmn.cfm?ID=K180599).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Velacur,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf22/K223287.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, 2268 Next Generation (2268NextGen),Department of Veterans Affairs,VA,OSVA: Office of the Secretary,Other,This system supports Federal contracting.,Inputting large amounts of data from paper forms into a digital system using AI.,"We receive information via a PDF VA Form. AI builder, a Microsoft product, “reads” the PDF VA Form and the information is transferred to a D-365 CRM database. This transfer of information from the PDF document to the database minimizes the need for human intervention and typographical errors. ",The AI Builder provides digital data from a PDF Document.,Implementation and Assessment,Neither,5/10/2022,4/11/2023,10/2/2023,,,,,,,,,,,,,,,,,,,,Yes,VEMS ID: 1018,,,,,,,,,,,,,,,,,,,,,,,,,,, Syngo Application Software,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf17/K170747.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Operationalizing Coronary Artery Calcium Computer Vision Model,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,This AI model's purpose is to screen Chest CT scans for coronary artery calcification and notify physicians of patients with increased cardiovascular risk. This will help physicians detect cardiovascular disease earlier and better treat their patients.,The model will output a coronary artery calcium score and also output image masks showing physicians where the model detected coronary artery calcium on the patients chest images.,Acquisition and/or Development,Both,1/1/2023,5/5/2023,,,Developed in-house.,,Yes,Veterans Health Administration,,No,,Yes,Yes,Yes,,"VA Imaging data, VINCI-CDW based data","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,"ARCHES, VA Long Beach AIOC",Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,,,,,,,,,,,,,,,,,, Rapid AI,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf23/K233582.pdf).,Operation and Maintenance,Both,5/13/2023,5/13/2023,5/13/2023,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, "Spine Planning (2.0), Elements Spine Planning, Elements Planning Spine",Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf22/K223553.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, VRS Employment Support,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Education & Workforce,,Improving the quality of written communications using AI tools.,We are using ChatGPT to support veterans in resume development as well as mock interview question if the veteran is interested.,The output is resumes and mock interview questions.,Operation and Maintenance,Neither,5/1/2023,5/1/2023,5/1/2023,,,,,,,,,,,,,,,,,,,,No,,,,,,,,,,,,,,,,,,,,,,,,,,,, Classifying the reportability of and extracting cancer diagnosis information from pathology reports or other clinical documents ,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,Searching for information using AI.,"The intended purpose is to highlight and show the extracted information by the system along with the original context for review by the nurse coordinators, within the health informatics tools currently in production, in the National Oncology Program Office. This will decrease the nurse coordinators’ information gathering time and improving their information efficiency. Information may also help identify Veterans with diagnoses of interest for review by the coordinator allowing for additional resources to be activated to support the Veteran if they meet program criteria. We do not anticipate this AI to be either safety impacting or rights impacting. The output does not serve as a principal basis for a decision. Coordinators are expected to review the output and corroborate its veracity using source information obtained from the EHR. The system resides completely within approved VA IT boundaries that are fully compliant with FISMA and HIPAA rules with availability and reliability set by OI&T. The information that the algorithm uses for input and its output relates to patient and disease features related to cancer diagnoses and therefore are unlikely to perpetuate inequalities related to protected characteristics. They are unrelated to educational benefits, housing grants, loans, risk assessment for insurance, job training programs, employment services, vocational rehabilitation, or hiring. It does not impact claims and benefits processing. While the system is envisioned to increase the efficiency of coordinators identifying information elements related the Veteran’s cancer diagnoses, these elements are not envisioned to be related to race, nor creating or promoting disparities. On the contrary, by improving coordinator efficiency, we believe the system will allow more Veterans to be supported; decreasing any existing inequities. It will also allow Coordinators be spend less time gathering information and more time interacting with Veterans, improving their ability to connect with the Veteran, learn about them, and provide Veteran-centric support, including care coordination, tracking, providing access to clinical trials if the Veteran is interested per standard of care policy set by the National Oncology Program Office, or provide other types of support. The system will make it easier for Coordinators to locate information about Veterans related to their diagnoses, however it will not be used for diagnosing disease, recommending treatment plans nor as a tool to monitor a patient’s health. It will not be used for health assessments or interventions, manage or dispense medications, or to automate labs. It will not make any decisions for the provider, but will make it easier for the provider to identify information elements of interest in the course of supporting Veterans. The system will also be entirely redundant, meaning Coordinators will still use information from the EHR to perform their workflows, both to find information about Veterans and to document their interactions. Coordinators will be able to fall back to EHR workflows should the system is unavailable for any reason. ","Binary label of reportability, i.e. if there is cancer diagnosis in the report. This will be highlighted in the original context and output along with the original context for nurse coordinators to review. Cancer diagnosis information including histology, site, and subsite. This will be highlighted in the original context and output along with the original context for nurse coordinators to review.",Acquisition and/or Development,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Post discharge 30-day readmission or death prediction model to select patients for expedited postdischarge clinic follow-up.,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,"The purpose of this AI is to act as an aid not a replacement for clinical decision-making about whether a patient should have clinical follow-up in the week following discharge (an accelerated timeframe) or follow-up in 2-4 weeks (the regular timeframe). The AI will consider dozens of factors to which a clinician does not have ready access or time to review during a typical discharge process. Patients that are predicted to have an adverse outcome will be recommended a quick follow-up but this will not be mandated, similarly patients not predicted to have an adverse outcome can still be scheduled for quick post discharge follow-up. Predicting which patients will die or be readmitted after discharge is a famously difficult unsolved clinical problem, and this tool will be one more data point for clinicians to aid but not replace clinical judgment during the discharge process. The AI’s intended benefit is to make our healthcare system more systematic and accurate in identifying high-risk patients at discharge as a primary goal. An aspirational benefit is that this process of more systematic more accurate follow-up for high risk patients will lead to decreased re-admissions and mortality after discharge.",The XG boost system outputs a 0 and 100% probability of readmission or death within 30 days.,Acquisition and/or Development,Both,6/1/2023,6/1/2023,,,Developed in-house.,,Yes,Veterans Health Administration,,No,Not applicable,No,No,Yes,,The AI training data set is our healthcare systems admissions and subsequent readmissions or death at 30 days between the fiscal years 2018 and 2023. This was derived from CDW data.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.","[""Marital Status"",""Age""]",,Yes,"Yes – agency has access to source code, but it is not public.",,No,,More than 12 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Using Artificial Intelligence to Predict Suicide,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,Searching for information using AI.,"Prevention of Veteran suicide is VHA’s top clinical priority. We use AI to identify patients with two key risk factors for death by suicide: 1) having access to a gun, and 2) having an active opioid use disorder. This project is funded by and developed with input from the Office of Suicide Prevention. THREE REASONS KNOWING ABOUT ACCESS TO GUNS IS CRITICAL TO SUICIDE PREVENTION First, when a person attempts suicide using a gun, the death rate is about 85%. For the most common method of attempting suicide—drug overdose—the death rate is less than 5%. Second, Veterans are more likely to own a gun and know better how to use it than other patients, so Veterans’ death rate when attempting suicide using a gun is even higher. Third, there are things clinicians can do to protect Veterans from their guns while they are suicidal, such as counselling them to give the guns or the ammunition to friends or family until they feel better. This is always voluntary, but Veterans very often agree to it, because they see and fear the risk. WHY KNOWING WHICH PATIENTS HAVE ACTIVE OPIOID USE IS CRITICAL TO SUICIDE PREVENTION Patients who die by suicide almost always have a prior history of emotional and mental health difficulties. When they die by suicide, it’s usually not due to that history, but rather because some bad life event (such as breaking up with their spouse) happens and then they have an overwhelming impulse to end their lives. Opioid use disorder (OUD) contributes to suicide risk in two ways. First, it makes bad life events much more common. It’s hard to keep a job or have good relationships with OUD. Second, OUD also increases impulsivity. That is, among patients who have just lost their jobs, those with OUD are more likely to have the impulse to end their lives. (Among substance use disorders, we focus on OUD because it is very common and there are good treatments available.) WHY USE AI TO DETERMINE ACCESS TO GUNS? The need for AI arise from how information is stored in an electronic health record. For historical reasons, when a clinician sees a patient, there are codes they can use to record what diseases (e.g., diabetes) the patient has, but there is no code for having access to a gun. VHA has considered asking all patients during intake for each visit whether they have access to a gun, but this is time consuming and can cause concern among patients about why they’re always being asked about guns. AI, specifically natural language processing (NLP) of clinical notes, offers another way to know which patients have guns. We have developed NLP algorithms that are very good at assessing from notes whether a patient owns or has access to a gun. But we use only notes already written by clinicians, any mention of gun access came voluntarily from the patient. However, to find this information, we need a method that can go through notes quickly, and NLP provides that method. WHY USE AI TO ASSESS WHETHER PATIENTS HAVE SUBSTANCE USE DISORDERS? OUD is a disease, and there are disease codes available to record it. However, because there are many different states you can be in with OUD (e.g., actively using vs in rehab vs sober for years), there are over 100 possible ways to code OUD. Most clinical settings (like outpatient clinics and ERs) have short list of commonly used codes for clinicians to pick from, but that may not offer up much specificity. So most clinics will only put on their list a single code, “opioid use, unspecified” and leave off all the more specific codes. Thus, typical coding practice does not differentiate between a patient who is currently using opioids or one who is 10 years sober (and, if you live along time with OUD, it is usually because you spend much of that time sober). That differentiation is important because the increase in impulsivity with OUD is only during active opiate use. NLP can again be useful, because descriptions of OUD in clinical notes are likely to be much more specific than the codes used","OUR INPUT, OUTPUT, AND INTENDED USER In the “Please add anything else you would like to share” box below we describe in more detail why it is necessary to use AI to find information about gun access or OUD (versus either not having the information or getting that information without using AI). We also describe how we build and test AI models to determine whether a patient has gun access or is actively using opioids. First, however, we think it is important to describe how human clinicians in VHA will use our AI. The INPUT to our AI models is simply the notes that have already been written on patients. These only contain information volunteered by the patient. The OUTPUT of our models will be simply indicators that a patient **may** have access to a gun or **may** have an active substance use disorder, along with a presentation of the text that leads the AI algorithm to reach its conclusion. However, the INTENDED USE of the output is that a human mental health clinician be notified about patients at risk, then read the note or notes that led to the determination that the patient was at risk. The clinician may conclude that the AI was incorrect (this occurs about 12-16% of the time with our current models) and drop the issue. Alternatively, the clinician may decide that the patient does have the risk factor and that the risk factor needs to be addressed. There is no planned interaction between the AI and the patient, nor does the AI make any clinical decisions. Rather, it is a tool to help point clinicians to potentially critical information that they might not otherwise have been aware of. OUR AI METHOD We begin by creating a list of common terms centered around gun access or OUD. We then pull thousands of pieces of clinical text with these terms in them, and have physicians determine whether the risk factor is present or not. (A term can be present, but the patient can still not have the risk factor in a variety of ways. It could be that “the patient’s brother has a gun” or “the patient has NO gun” or “the patient used to use Fentanyl, but quit”.) The text that has been assessed by physicians is called “labeled data”. We then give our NLP models some of the labeled data, so they can learn what positive and negative examples look like. For each risk factor, we develop 4 non-neural AI text classification models: random forest, bagging, gradient boosting, and logistic regression with ridge penalization. We also implement two large language models, BioBERT and Bio-ClinicalBERT. Subsequently, we test each algorithm on other data for which we have (but don’t give the algorithm) the label and ask each algorithm to produce its own labels. We then compare each algorithm’s conclusions to the physicians’ conclusions. That is how we know that our algorithms so far, will be incorrect about 12-16% of the time. Over time, we hope to improve the performance of these algorithms. For example, the large language models are relatively new additions to the AI toolkit and perform better than the non-neural models. We expect further advances over time. However, the current level of accuracy is already higher than many commonly used diagnostic tests, such as mammograms or erythrocyte sedimentation rates. WHERE DO WE DO THIS WORK? Our team currently uses the Summit Data Platform (SDP) – formerly called the Health Data and Analytics Platform (HDAP) – maintained by the VA Office of Information Technology. This is an Azure Machine Learning environment.",Acquisition and/or Development,Both,10/1/2021,10/1/2021,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, RayStation 11B,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf22/K220141.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Identify access to genetic testing in Veterans with breast cancer. ,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,Searching for information using AI.,"Stakeholders in the National Oncology Program that are required to understand quality of standard of care access provided by VA to Veterans with cancer to germline genetic testing. Veterans who have not been provided standard of care access may be provided additional support in the form of care coordination services to ensure they receive standard of care access. The AI use case is intended to help the Stakeholders in the National Oncology Program understand quality of standard of care access provided by VA to Veterans with cancer to germline genetic testing. This will help stakeholders be able to identify which veterans have not been provided with germline genetic testing access more efficiently and at almost real time, thus being able to provide additional support in the form of care coordination services to ensure they receive standard of care access. We do not anticipate this AI to be either safety impacting or rights impacting. The output does not serve as a principal basis for a decision. Coordinators are expected to review the output and corroborate its veracity using source information obtained from the EHR. The system resides completely within approved VA IT boundaries that are fully compliant with FISMA and HIPAA rules with availability and reliability set by OI&T. The information that the algorithm uses for input and its output relates to patient , disease features, and whether genetic testing is offered are related to cancer diagnoses and therefore are unlikely to perpetuate inequalities related to protected characteristics. They are unrelated to educational benefits, housing grants, loans, risk assessment for insurance, job training programs, employment services, vocational rehabilitation, or hiring. It does not impact claims and benefits processing. While the system is envisioned to increase the efficiency of coordinators identifying information elements related the Veteran’s cancer diagnoses, and whether standard of care access to genetic testing was offered to Veterans, these elements are not envisioned to be related to race, nor creating or promoting disparities. On the contrary, by improving coordinator efficiency and awareness of whether Veterans received the care adherent to VA NOP guidelines, we believe the system will allow more Veterans to be supported in adherence with NOP guidelines; decreasing any existing inequities. It will also allow Coordinators be spend less time gathering information and more time interacting with Veterans, improving their ability to connect with the Veteran, learn about them, and provide Veteran-centric support, including facilitating access to genetic testing per NOP guidelines. The system will make it easier for Coordinators to locate information about Veterans related to their diagnoses, however it will not be used for diagnosing disease, recommending treatment plans nor as a tool to monitor a patient’s health. It will not be used for health assessments or interventions, manage or dispense medications, or to automate labs. It will not make any decisions for the provider but will make it easier for the provider to identify information elements of interest in the course of supporting Veterans. The system will also be entirely redundant, meaning Coordinators will still use information from the EHR to perform their workflows, both to find information about Veterans and to document their interactions. Coordinators will be able to fall back to EHR workflows should the system is unavailable for any reason. ","Binary output of Yes or no on if the patients have access to germline genetic testing, and the reason of the output. This will be highlighted in the original context and output along with the original context for nurse coordinators to review.",Acquisition and/or Development,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Classify clinical pathway for cancer patients,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,Searching for information using AI.,"The VA National Oncology Program has invested in creating clinical pathways that guide providers to provide high quality evidence-based standard of care treatments to Veterans with cancer. However at this time, it is unclear how many or how often providers are ‘on pathway’ or are concordant with this recommendation. Work will focus on leveraging AI to assess degree of adoption of various pathways. For example: To assess care quality across the VA in regards to genetic testing in Veterans with prostate cancer, and provide the needed information to engage in quality improvement efforts, the VA NOP proposes to develop, evaluate and implement an informatics workflow that is capable of extracting information regarding whether a Veteran with prostate cancer has been offered access to germline and somatic genetic testing. Work will follow the framework NOP developed to assess germline genetic testing in Veterans with breast cancer in response to the Advances in Mammography and Medical Options (MAMMO) for Veterans Act of 2022. Briefly, Veterans receiving care for prostate cancer in VA will be identified using electronic health information in the VA Corporate Data Warehouse, and their latest relevant clinical notes will be retrieved. Data from VistA CPRS prostate cancer pathway templates will provide a gold standard for developing and evaluating the feasibility of using an open-source generative large language model to determine whether a Veteran has been offered genetic testing according to clinical notes. Earlier work has been successful using zero-shot (or few-shot) prompt development. Model fine tuning can be considered should these approaches fail. Upon achieving acceptable accuracy, the model will then be implemented for near real time determination of access to genetic testing at scale, especially in sites where this information is not available. This work will allow assessment of quality of care as it relates to adherence to VA endorsed clinical pathways on genetic testing in Veterans with prostate cancer. It will also facilitate efforts for improving care delivered through identification of sites that can be supported through existing efforts such as the National Tele-oncology Office, or cancer care navigation and tracking initiatives. We do not anticipate this AI to be either safety impacting or rights impacting. The output does not serve as a principal basis for a decision. Stakeholders are expected to review the output at the population level, increasing their awareness of how the system as a whole is attending to Veteran needs per program office policy. Any use of the system to address needs at the site level or Veteran level will involve human review and corroboration of the output veracity using source information obtained from the EHR. The system resides completely within approved VA IT boundaries that are fully compliant with FISMA and HIPAA rules with availability and reliability set by OI&T. The information that the algorithm uses for input and its output relates to documentation of patient, disease, and prior treatment features related to cancer diagnoses and therefore are unlikely to perpetuate inequalities related to protected characteristics. They are unrelated to educational benefits, housing grants, loans, risk assessment for insurance, job training programs, employment services, vocational rehabilitation, or hiring. It does not impact claims and benefits processing. While the system is envisioned to increase the effectiveness and efficiency of stakeholders identifying information elements related the Veteran’s cancer diagnoses and treatment, these elements are not envisioned to be related to race, nor creating or promoting disparities. On the contrary, by improving identification of off pathway care, we believe the system will offer the opportunity for stakeholders to identify areas where Veterans, or systems in need more support, decreasing any existing inequities. It will not be used for diagnosing disease","Class for clinical pathway, i.e., which clinical pathway the cancer patient is on. This will be highlighted in the original context and output along with the original context for nurse coordinators to review.",Acquisition and/or Development,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Enterprise Command Center - Generative AI ,Department of Veterans Affairs,VA,OIT: Office of Information & Technology,Mission-Enabling (internal agency support),,None of the above.,The benefit of this use case is to provide plain English alert messages to service owners. The current alert language has proven suboptimal and results in a inordinate volume of alerts not being actioned. By feeding alerts into an LLM the message can be reconfigured to be understood by a human who in turn can take appropriate steps to resolve the alert.,Human readable alert messages,Initiated,Neither,3/22/2024,,,,Developed in-house.,,No,,,No,,No,Yes,Yes,,"ServiceNow, SolarWinds, Splunk, Dynatrace, AppDynamics, and ScienceLogic all host alert data that would be used to train the model.","Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,No,No – agency does not have access to source code.,,No,"2879, 2462, 2464, 2818, 2463",6-12 months,Yes,,No,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Extract patients’ diagnosis information and map the diagnosis to oncotree,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,Searching for information using AI.,"Patients’ diagnosis information has only been extracted in the format of ICD0 code, although clinicians are only able to use the diagnosis information that is described according to the oncotree in situations such as clinical trial matching. We aim to use the AI system to identify the diagnosis information according to oncotree. This will help highlight the diagnosis information according to oncotree on the clinical documents that the nurse navigators could review more quickly, therefore help the document review more efficiently. We do not anticipate this AI to be either safety impacting or rights impacting. The output does not serve as a principal basis for a decision. Coordinators are expected to review the output and corroborate its veracity using source information obtained from the EHR. The system resides completely within approved VA IT boundaries that are fully compliant with FISMA and HIPAA rules with availability and reliability set by OI&T. The information that the algorithm uses for input and its output relates to patient and disease features related to cancer diagnoses and therefore are unlikely to perpetuate inequalities related to protected characteristics. They are unrelated to educational benefits, housing grants, loans, risk assessment for insurance, job training programs, employment services, vocational rehabilitation, or hiring. It does not impact claims and benefits processing. While the system is envisioned to increase the efficiency of coordinators identifying information elements related the Veteran’s cancer diagnoses, these elements are not envisioned to be related to race, nor creating or promoting disparities. On the contrary, by improving coordinator efficiency, we believe the system will allow more Veterans to be supported; decreasing any existing inequities. It will also allow Coordinators be spend less time gathering information and more time interacting with Veterans, improving their ability to connect with the Veteran, learn about them, and provide Veteran-centric support, including care coordination, tracking, providing access to clinical trials if the Veteran is interested per standard of care policy set by the National Oncology Program Office, or provide other types of support. The system will make it easier for Coordinators to locate information about Veterans related to their diagnoses, however it will not be used for diagnosing disease, recommending treatment plans nor as a tool to monitor a patient’s health. It will not be used for health assessments or interventions, manage or dispense medications, or to automate labs. It will not make any decisions for the provider, but will make it easier for the provider to identify information elements of interest in the course of supporting Veterans. The system will also be entirely redundant, meaning Coordinators will still use information from the EHR to perform their workflows, both to find information about Veterans and to document their interactions. Coordinators will be able to fall back to EHR workflows should the system is unavailable for any reason. ","Patients’ diagnosis according to oncotree, and this will be highlighted in the original context and output along with the original context for nurse coordinators to review.",Acquisition and/or Development,Neither,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Adobe Creative Cloud ,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,Creating visually appealing presentations using AI-driven design suggestions.,Develop nursing education presentations ,Used to make animations or to populate AI art.,Operation and Maintenance,Neither,,,,,,,,,,,,,,,,,,,,,,,Yes,ASMS ID: 1279,,,,,,,,,,,,,,,,,,,,,,,,,,, QLAB Advanced Quantification Software,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf20/K200974.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, "Identify cancer patients symptom burden, including symptoms supported by embedded mental health. ",Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,Searching for information using AI.,"Mental Health is important for cancer patients and could be barrier for effective and timely cancer care. Approximately one-third of persons with cancer will develop a mood or anxiety disorder. Mental health of Veterans with cancer is a priority for the VA National Oncology Program. We have been tasked to develop and implement a health information technology tool that will facilitate the effectiveness and efficiency of mental health providers embedded in cancer clinics, such as the one embedded in the Oncology clinic in the Durham VA. One of the challenges he faces is to identify patients that are most likely to benefit from the intervention. We aim to use AI to extract patients’ symptom burden information including the symptoms related to mental health needs from clinical notes and surface them for the embedded mental health provider. National oncology program embedded mental health specialists and nurse coordinators will review the information, allowing them to efficiently identify Veterans that may benefit from additional support for symptom management or mental health. This could allow embedded mental health specialists to monitor cancer patients among veterans at a higher risk level more closely and provide the most appropriate level of care. It could also potentially help veterans have increased cancer treatment engagement and quality of care, which are essential for cancer patients’ disease prognosis. ","Information on patient symptom burden including symptoms related to mental health needs of cancer patients, this will be highlighted and output along with the clinical documents that are used to extract the information for the nurse coordinators to review. We do not anticipate this AI to be either safety impacting or rights impacting. The output does not serve as a principal basis for a decision. Coordinators are expected to review the output and corroborate its veracity using source information obtained from the EHR. The system resides completely within approved VA IT boundaries that are fully compliant with FISMA and HIPAA rules with availability and reliability set by OI&T. The information that the algorithm uses for input and its output relates to patient and disease features related to cancer diagnoses as well as descriptions of symptom burden and therefore are unlikely to perpetuate inequalities related to protected characteristics. They are unrelated to educational benefits, housing grants, loans, risk assessment for insurance, job training programs, employment services, vocational rehabilitation, or hiring. It does not impact claims and benefits processing. While the system is envisioned to increase the efficiency of providers identifying information elements related the Veteran’s cancer diagnoses and symptom burden documented in the EHR, these elements are not envisioned to be related to race, nor creating or promoting disparities. On the contrary, by improving efficiency, we believe the system will allow more Veterans to be supported, decreasing any existing inequities. It will also allow providers spend less time gathering information and more time interacting with Veterans, improving their ability to connect with the Veteran, learn about them, and provide Veteran-centric support. The system will make it easier for providers to locate information about Veterans related to their symptom burden, however it will not be used for diagnosing disease, recommending treatment plans nor as a tool to monitor a patient’s health. It will not be used for health assessments or interventions, manage or dispense medications, or to automate labs. It will not make any decisions for the provider but will make it easier for the provider to identify information elements of interest while supporting Veterans. The system will also be entirely redundant, meaning providers will still use information from the EHR to perform their workflows, both to find information about Veterans and to document their interactions. Providers will be able to fall back to EHR workflows should the system is unavailable for any reason.",Acquisition and/or Development,"Rights-Impacting ",,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Ambulatory heart rhythm monitoring,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,Searching for information using AI.,AI is used to analyze 14 days of heart rhythm to narrow down periods of abnormalities and identify such rhythms.,Its a report with various abnormalities identified and listed for the clinician to interpret and make recommendations.,Operation and Maintenance,Both,,,,,,,,,,,,,,,,,,,,,,,No,,,,,,,,,,,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Insights Engine,Department of Veterans Affairs,VA,VEO: Veterans Experience Office,Mission-Enabling (internal agency support),,Summarizing the key points of a lengthy report using AI.,"The Insights Engine provides a human-language interface to query aggregate-level / anonymized customer experience and operational metrics and data in support of VA research and design initiatives. This tool leverages large language model interpretation and agents to write queries for structured and unstructured data, and articulate responses based on relevant source review or Retrieval Augmented Generation (RAG). This tool will serve as a research librarian to empower research and design teams across the VA enterprise to identify and leverage past-relevant learnings that reduce redundant research initiatives and build on prior work and knowledge. Staff can inquire about what has been done / is known about x, y, z and receive answers in plain language.","Prior research findings, qualitative and quantitative insights.",Acquisition and/or Development,Neither,4/1/2024,4/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, "Philips EPIQ Diagnostic Ultrasound System, Philips Affiniti Diagnostic Ultrasound System",Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf21/K212704.pdf).,Operation and Maintenance,Both,11/28/2022,11/28/2022,11/28/2022,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Glassbeam Clinsights ,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,Identifying unusual patterns in system logs from a single incident report using AI.,"The benefits of this AI is to gather medical device data, and use machine knowledge to attempt prediction of when medical equipment may fail. ","Gathers medical device hardware information and compares the data that was gathered to determine if it is in normal range, declining, or failed to alert the technicians as soon as possible to repair/replace faulty parts before they are used in a medical procedure.",Implementation and Assessment,Both,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, VX Insights Hub Agent,Department of Veterans Affairs,VA,VEO: Veterans Experience Office,Mission-Enabling (internal agency support),,Summarizing the key points of a lengthy report using AI.,"The VX insights hub provides a human-language interface to query aggregate-level / anonymized customer experience and operational metrics and data in support of VA performance improvement and decision-making. This tool leverages large language model interpretation and agents to write queries for structured and unstructured data, and articulate responses based on relevant source review or Retrieval Augmented Generation (RAG). This tool will empower VA staff in decision-making and performance improvement roles, who may not have a background in data science, statistics or analytics, to inquire about performance and experience and receive answers in plain language.","Natural language responses to CX and operations questions + validation information about the query that was written and run, etc.",Acquisition and/or Development,Neither,2/1/2024,3/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, "OPTIS Mobile Next Imaging System, OPTIS Integrated Next Imaging System",Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf21/K210458.pdf).,Operation and Maintenance,Both,7/17/2023,7/17/2023,7/17/2023,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Automated Decision Support,Department of Veterans Affairs,VA,VBA: Veterans Benefits Administration,Government Services (includes Benefits and Service Delivery),,None of the above.,"Intake of VA Claims materials as well as other Veteran, VSO, and stakeholder submissions results in contentions which are prescribed as eligible. The platform ingests and takes initial actions on claims and contentions (largely those impacted by the PACT Act) which include medical record retrieval, review of the VBMS eFolder, generates a summary review document, drafting or submitting contract examination requests via robotics process automation (RPA), and/or changing the claim status to Ready For Decision (RFD). ","Utilizing prescribed machine learning and Natural Language Processing (NLP), ADS utilizes Intelligent Form Recognition (IFR), Optical Character Recognition (OCR), Intelligent Character Recognition (ICR), and NLP to extract an average of 95 fields on over 1500 form layouts within VA’s purview. The system generates a summary review document, drafting or submitting contract examination requests via robotics process automation (RPA), and/or changing the claim status to Ready For Decision (RFD).",Operation and Maintenance,Both,7/1/2021,8/1/2021,1/27/2022,,Developed with both contracting and in-house resources.,IDIQ 36C10E19D0018 APCAS 36C10D21N0013 MAS 36C10E20N0028 BTP 36C10D23N0010 ,Yes,Veterans Benefits Administration,,No,"Quoted below is a section from our white paper on ethical implementation of AI iaw the executive orders and recent legislation: Purposeful and performance-driven: IBM’s NLP services, when combined with OCR/ICR capabilities, enable our platform to interpret unstructured and semi-structured forms, correspondence, and other data types, including handwriting. To date, we have deployed 30+ custom, proprietary NLP models within the VA automation infrastructure, significantly improving upon out-of-the-box OCR capabilities by further extracting and classifying the data necessary for downstream actions. - Accurate, reliable, and effective: Applying NLP significantly improves our extraction quality across form types and variations and allows us to infuse intelligence into the business process. NLP processing enables us to uncover and reveal insights (e.g., identify critical “flash” conditions; linking Veteran written disabilities to a specific condition to include characterizing it as ‘secondary to’) into more complex processing determinations based on the information within the data. Cognitive AI models such as ML, vision, and speech are used throughout our solution to create functional capabilities that continuously improve over time. Safe, secure, and resilient: IBM’s solution resides upon a FedRAMP-certified Amazon Web Services (AWS) GovCloud environment and operates 24/7 within a current VA Authority to Operate (ATO). - Understandable: A foundation of ethical AI/ML techniques is explainability – that is, the ability to understand and explain how an automation arrived at a certain outcome. IBM’s approach to documenting requirements and our detailed processing audits provides the explainability necessary to provide for proper processing outcomes. - Regularly monitored: Supervision and Human-in-the-Loop (HIL) capabilities support immediate data correction and supervised learning to improve extraction accuracy. - Transparent and accountable: IBM’s current process is to provide detailed documentation to VBA prior to the release of new NLP models. This documentation details the purpose and output of the model, the training data and algorithms used, and the limitations and availability of the models. IBM works with VBA throughout the requirements, development, testing and post-production process to review the approach, models and methods used to meet their business requirements.",Yes,Yes,Yes,,NLP/ML has primarily focused around contents within the eFolder. Please see the MOU/iSA within eMass and Vasi as the 46 page document delineates data optimization and boundaries.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,VBAAP - Vasi 2522 eMASS 1143,Less than 6 months,No,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Compliance Made Easy (CME) Cloud Optimization,Department of Veterans Affairs,VA,OIT: Office of Information & Technology,Other,"This may fall under government services, but this is directly related to supporting the office of the executive secretary with official communications.",Improving the quality of written communications using AI tools.,"The purpose is to provide automated review of official communications against the approved VA style guide, tone, and other key elements to making communications to the public effectively. The goal is to reduce the amount of time needed to create and review the documents leading to saved time and money. The benefits of using AI include more intelligent control over a rule based system, low/no code updates to the model through direct NLP prompts to update the approved style guides, leveraging machine learning to improve communications and tone over time, less reliance on human intervention to review and edit communications. ","AI outputs will include updated communications style and formatting based on the style guide, suggested language to improve tone, readability and conciseness of communications, and data based on the model running.",Initiated,Neither,9/15/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, O-arm O2 Imaging System,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf24/K240465.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Mail Automation Services,Department of Veterans Affairs,VA,VBA: Veterans Benefits Administration,Government Services (includes Benefits and Service Delivery),,None of the above.,"Mail Automation Services (MAS) is a ITTT service for intake of VA Claims materials as well as other Veteran, VSO, and stakeholder submissions. The platform ingests and reviews 25k to 40k packets of information per day, primarily derived from the Centralized Mail Portal. Utilizing prescribed machine learning and Natural Language Processing (NLP), MAS utilizes Intelligent Form Recognition (IFR), Optical Character Recognition (OCR), Intelligent Character Recognition (ICR), and NLP to extract an average of 95 fields on over 1500 form layouts within VA’s purview.",VBA End-Product establishment and correct business line orientation for ingested submissions are the most common outcomes and outputs for the MAS platform which runs under Veterans Benefits Administration Automation Platform (VBAAP).,Operation and Maintenance,"Rights-Impacting ",7/1/2019,10/1/2019,1/6/2020,,Developed with both contracting and in-house resources.,IDIQ 36C10E19D0018 APCAS 36C10D21N0013 MAS 36C10E20N0028 BTP 36C10D23N0010 ,Yes,Veterans Benefits Administration,,No,"Quoted below is a section from our white paper on ethical implementation of AI iaw the executive orders and recent legislation: Purposeful and performance-driven: IBM’s NLP services, when combined with OCR/ICR capabilities, enable our platform to interpret unstructured and semi-structured forms, correspondence, and other data types, including handwriting. To date, we have deployed 30+ custom, proprietary NLP models within the VA automation infrastructure, significantly improving upon out-of-the-box OCR capabilities by further extracting and classifying the data necessary for downstream actions. - Accurate, reliable, and effective: Applying NLP significantly improves our extraction quality across form types and variations and allows us to infuse intelligence into the business process. NLP processing enables us to uncover and reveal insights (e.g., identify critical “flash” conditions; linking Veteran written disabilities to a specific condition to include characterizing it as ‘secondary to’) into more complex processing determinations based on the information within the data. Cognitive AI models such as ML, vision, and speech are used throughout our solution to create functional capabilities that continuously improve over time. Safe, secure, and resilient: IBM’s solution resides upon a FedRAMP-certified Amazon Web Services (AWS) GovCloud environment and operates 24/7 within a current VA Authority to Operate (ATO). - Understandable: A foundation of ethical AI/ML techniques is explainability – that is, the ability to understand and explain how an automation arrived at a certain outcome. IBM’s approach to documenting requirements and our detailed processing audits provides the explainability necessary to provide for proper processing outcomes. - Regularly monitored: Supervision and Human-in-the-Loop (HIL) capabilities support immediate data correction and supervised learning to improve extraction accuracy. - Transparent and accountable: IBM’s current process is to provide detailed documentation to VBA prior to the release of new NLP models. This documentation details the purpose and output of the model, the training data and algorithms used, and the limitations and availability of the models. IBM works with VBA throughout the requirements, development, testing and post-production process to review the approach, models and methods used to meet their business requirements.",Yes,Yes,Yes,,NLP/ML has primarily focused around contents submitted via the centralized mail portal. Please see the MOU/iSA within eMass and Vasi as the 46 page document delineates data optimization and boundaries.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,VBAAP - Vasi 2522 eMASS 1143,Less than 6 months,No,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, NeuroQuant,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf17/K170981.pdf).,Operation and Maintenance,Both,10/31/2017,10/31/2017,10/31/2017,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Master Claims Assistance Tool (M-CAT),Department of Veterans Affairs,VA,VBA: Veterans Benefits Administration,Government Services (includes Benefits and Service Delivery),,Searching for information using AI.,"Function of the system is generative AI use and derivatives to assist VA personnel. Inputs to the model include policy, procedure, and other knowledge management documents such as CFR database knowledge and user entered inquiries. ",Outputs include RAG model results from chatbot. Intended users are VA or VBA staff as directed by SES.,Acquisition and/or Development,"Rights-Impacting ",8/1/2024,8/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, MAGNETOM Vida; MAGNETOM Lumina; MAGNETOM Aera; MAGNETOM Skyra; MAGNETOM Prisma; MAGNETOM Prisma fit,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfPMN/pmn.cfm?ID=K231560).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, 3D Quorum ,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,Mammography. AI-powered analytics - Generates 6mm 'SmartSlices' from the original high-resolution 3D data. Identifies clinically relevant regions of interest through overlapping slices (3mm) to ensure no loss of 3D data. Reduces the number of 3D images to be reviewed by the Radiologist by 2/3. ,Produces 6mm 'SmartSlices' from native 3D data to highlight clinically relevant or potentially clinically relevant regions of interest. Augments the decision making process of Board Certified or Board Eligible Radiologists. Interpreting Radiologists retain the ability to 'accept' or 'reject' the suggested outputs.,Operation and Maintenance,Both,1/1/2019,1/1/2020,1/1/2021,,,,Yes,Veterans Health Administration,,No,"This use case doesn't appear to meet the definition in M-19-15 as disseminating 'influential scientific, financial, or statistical information.'",Yes,No,No,,This use case will access PHI/PII from VA 'Home of Record' systems (Vista/CPRS) and peripheral Radiology PACS & Dictation Systems.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,No – agency does not have access to source code.,N/A,No,,More than 12 months,Yes,,No,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, INtuition-Structural Heart Module,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfPMN/pmn.cfm?ID=K191585).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Intelligent 2D ,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,"Mammography - Produces synthetic 2D images from 3D mammography data to improve the detection of breast lesions while reducing radiation dose. Reduces the 'noise' if the images, helping to eliminate random elements that may distort the image. Provides Interpreting Radiologists improved detection ability through reduction of artifacts that may reduce visibility of lesions - ultimately providing a clearer picture of breast tissue. FDA Approved. ",Provides synthetic 2D image to augment the Interpreting Radiologist's ability to detect breast lesions.,Operation and Maintenance,Both,1/1/2019,1/1/2020,1/1/2021,,,,Yes,Veterans Health Administration,,No,"This use case doesn't appear to meet the definition in M-19-15 as disseminating 'influential scientific, financial, or statistical information.'",Yes,No,No,,This use case will access PHI/PII from VA 'Home of Record' systems (Vista/CPRS) and peripheral Radiology PACS & Dictation Systems.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,No – agency does not have access to source code.,,No,,More than 12 months,Yes,,No,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, QV CAD,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,Automated Breast Ultrasound (ABUS) solution from QView Medical. Aids Radiologists with interpreting screening procedures for women who have negative mammograms but have had a 3D breast ultrasound due to dense breast tissue. Receives input images via standard DICOM format. Native imaging from the ABUS together with the output of the QVCAD images are displayed on the viewer. QVCAD engine uses several image patter recognition processes and artificial neural networks to detect suspicious areas in the breast with the objective to distinguish potential breast lesions from normal breast tissue. ,Detects suspicious or potentially suspicious areas in the breast. Augments the decision making process of Board Certified or Board Eligible Radiologists. Interpreting Radiologists retain the ability to 'accept' or 'reject' the suggested outputs.,Acquisition and/or Development,Both,10/1/2023,9/6/2024,,,,,Yes,Veterans Health Administration,,No,"This use case doesn't appear to meet the definition in M-19-15 as disseminating 'influential scientific, financial, or statistical information.'",Yes,No,No,,This use case will access PHI/PII from VA 'Home of Record' systems (Vista/CPRS) and peripheral Radiology PACS & Dictation Systems.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,No – agency does not have access to source code.,N/A,No,,More than 12 months,Yes,,No,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, FlexLine: Detect patterns and make corrective action recommendations in application and dependency time to mitigation data. ,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Mission-Enabling (internal agency support),,None of the above.,"Within the FlexLine materials management application, this AI capability would look at patterns across the time-to-mitigation data and potentially other scans for application vulnerabilities. If teams are not resolving vulnerabilities within the 30, 45, 60 day periods, then it could expose our products and degrade Veteran care. The goal would be to connect the duration to mitigate to other available data trends and recommend actions to the application teams that could help them resolve the issues faster.",Will provide detailed text data hat combines/correlates scan results for the app team and what might be causing the inability to resolve. This additional intelligence and recommended resolution(text) could be provided to the app team through automated Jira tickets to help them understand a way to resolve the issue.,Initiated,Neither,9/19/2024,,,,Developed with both contracting and in-house resources.,"36C10G24D0048, 36C10G24N0112",No,,,No,Not Applicable,No,Yes,Yes,,OCC FlexLine vulnerability and scan data.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.","[""Other""]",No demographic variables are used. Scan result data and variables are not tied to individuals.,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,2313,,Yes,,Yes,,Other,"We've not made a request to the relevant data science groups as we are building resource provisioning, and MLOps capabilities first.",Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Medical Imaging Auto-segmentation,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,Searching for information using AI.,To make the process of drawing regions of interest more efficient ,Rather than a physician manually outlining a lesion or organ such as the liver the AI system uses imaging data and anatomic atlas information to create a proposed outline that the physician can then adjust or approve for use,Operation and Maintenance,Both,,,,,,,,,,,,,,,,,,,,,,,Yes,MD-LITE ID: 2278,,,,,,,,,,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, FFRangio,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf19/K192442.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Quantra ,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,Mammography. Uses multi-class support vector machine (SVM) based classification technique to segregate breast types into four categories based on breast parenchymal tissue pattern and texture representation to assist Radiologists in interpretations and reduce intra- and inter-reader variability. ,Classifies breast type. Augments the decision making process of Board Certified or Board Eligible Radiologists. Interpreting Radiologists retain the ability to 'accept' or 'reject' the suggested outputs.,Operation and Maintenance,Both,1/1/2019,1/1/2020,1/1/2022,,,,Yes,Veterans Health Administration,,No,"This use case doesn't appear to meet the definition in M-19-15 as disseminating 'influential scientific, financial, or statistical information.'",Yes,No,No,,This use case will access PHI/PII from VA 'Home of Record' systems (Vista/CPRS) and peripheral Radiology PACS & Dictation Systems.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,No – agency does not have access to source code.,N/A,No,,More than 12 months,Yes,,No,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, VSTOne – AI-Driven Patient Monitoring and Care Delivery,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,Identifying unusual patterns in system logs from a single incident report using AI.,"Supporting Care Through AI The healthcare industry is at a critical juncture due to the convergence of rising patient demands and severe staffing shortage, specially in nursing. According to the U.S Bureau of Labor Statistics projection in 2022, American hospitals will be hiring 1,100,00 registered nurse positions. In response to this challenge AI driven patient monitoring care delivery can proactively assist nurses. The integration of artificial intelligence into healthcare presents a unique opportunity to enhance patient outcomes while supporting clinical staff in an increasingly challenging environment. With the looming shortage of nursing staff and the growing demand for patient care, there is an urgent need for technological solutions that support nurses in their roles. Today, I present VSTOne, an innovative AI solution designed to assist nurses, prevent patient falls, and improve care quality through continuous monitoring and real-time alerts.","Supporting Care Through AI VSTOne leverages advanced AI to provide comprehensive monitoring solutions that address several critical areas of patient care: • Assisting Nurses: Automated monitoring allows nurses to focus on higher-level care tasks. • Preventing Falls: The system predicts potential falls 30-65 seconds in advance, allowing timely intervention. • Preventing Pressure Ulcers: Early detection of pressure ulcer risks through continuous monitoring of patient movement. • Telemetry and Telehealth: Vitals are tracked via wearable patches, providing real-time data to clinicians, and enabling telehealth consultations.",Initiated,Both,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Surveillance and Reporting of Suicidal Ideation Assessment in PTSD Specialty Care Clinical Notes using Natural Language Processing (NLP),Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,We endeavor to surveil and monitor nationally both the conduct and quality of suicide ideation (SI) assessment during treatment in PTSD specialty care. We hope eventually to increase the consistency and quality of SI assessment in PTSD care to enhance clinical management of suicide risk. ,"Our methods entail the use of natural language processing (NLP) of progress note documentation of treatment within the PTSD Clinical Team (PCT) specialty care clinics, to identify SI mentions, to quantify SI assessment documentation, and to classify SI mention clinical quality. We will provide reports of consistency and quality and the national, VISN, and medical center levels.",Acquisition and/or Development,Neither,8/1/2023,10/1/2023,,,Developed in-house.,,No,,,No,N/A,Yes,Yes,Yes,,VA Corporate Data Warehouse (CDW) provides clinical progress notes in text form through its TIU for our use in this project.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.","[""Other""]","The NLP machine learning and rule-based processes rely solely on clinical progress note text to identify suicide ideation assessment, and do not utilize demographic variables in its machine learning/rule-based model.",Yes,"Yes – agency has access to source code, but it is not public.",,Yes,CDW,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, "FastStroke, CT Perfusion 4D",Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf19/K193289.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Koios,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,Used with Automated Breast Ultrasound (ABUS). Classifies breast ultrasound lesions and automates reporting for improved efficiency and streamlined workflows. Increased sensitivity and specificity with reduced variability. Reduces unnecessary treatments (existing BI-RADS 3's can be accurately and safely downgraded) and provides probability of malignancy aligned to BI-RADS categorization. ,Classifies breast ultrasound lesions. Augments the decision making process of Board Certified or Board Eligible Radiologists. Interpreting Radiologists retain the ability to 'accept' or 'reject' the suggested outputs.,Acquisition and/or Development,Both,10/1/2023,9/6/2024,,,,,Yes,Veterans Health Administration,,No,"This use case doesn't appear to meet the definition in M-19-15 as disseminating 'influential scientific, financial, or statistical information.'",Yes,No,No,,This use case will access PHI/PII from VA 'Home of Record' systems (Vista/CPRS).,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,No – agency does not have access to source code.,N/A,No,,More than 12 months,Yes,,No,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Evaluation of Autonomous Assistive Feeding System in Spinal Cord Injury Patients,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,"The AI use is to improve autonomous bite acquisition by robotic arm as well as to improve transfer to the participant's mouth. Inputs include video analysis of caretakers administering/transferring bite to participants, as well as a web interface where feedback from stakeholders will be collected to improve the robotic arm's actions and to provide additional contextual information for bite acquisition and transfer. This information will be used by the AI to improve the robotic arms autonomous functions of bite acquisition and transfer. The ultimate goal of the robotic arm function is to provide autonomous or semi-autonomous support to improve functional independence for patients with upper extremity weakness from spinal cord injury or other causes."," Inputs and Outputs: Our experimental setup will consist of bite acquisition trials under two settings. In the autonomous setting, the robot will acquire the food autonomously. In the shared autonomy setting, the robot may acquire the food autonomously, but during certain times, may query the end-user or an expert research assistant based on its confidence about its manipulation actions and use their feedback to complete the bite acquisition process. For both the settings, the robot will use our online learning framework to design its behaviors. In the shared autonomy setting, we will develop an accessible web interface that the stakeholders can use to provide feedback on what actions to use or what additional contexts may be useful for bite-acquisition",Acquisition and/or Development,Safety-impacting,5/13/2024,5/13/2024,,,Developed in-house.,,Yes,Veterans Health Administration,,No,,Yes,No,No,,Will obtain video and input data from trials of robotic feeding arm once research begins proper,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.","[""Other""]","data will be collected by video analysis, user/stakeholder input to continually refine the motions of the robotic arm used for bit acquisition and bite transfer to patient",Yes,No – agency does not have access to source code.,,No,,6-12 months,No,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, Using AI to read PDFs and parse the information to SharePoint list,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,Inputting large amounts of data from paper forms into a digital system using AI.,"The AI will read PDFs from the TPA, and parse the data into a sharepoint list, reducing the employees processing time by 20%","SharePoint, and will be used in Power BI.",Initiated,Neither,5/29/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Discovery MI Gen2,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf21/K211846.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Genius AI,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,Mammography. Assists the Radiologist in locating lesions likely representing breast cancer by searching each slice of the tomosynthesis image set. Lesions are marked on the appropriate slices that can be overlaid on a synthesized 2D image and on 3DQuorum SmartSlices. These 2D overlays augment Radiologists by providing an overview image with suspicious areas indicated with quick navigation to the tomosynthesis slice that the mark was originally identified on. ,Lesions are marked on identified imaging slices with reference back to corresponding tomosynthesis slice.,Operation and Maintenance,Both,1/1/2019,1/1/2021,1/1/2022,,,,Yes,Veterans Health Administration,,No,"This use case doesn't appear to meet the definition in M-19-15 as disseminating 'influential scientific, financial, or statistical information.'",Yes,No,No,,This use case will access PHI/PII from VA 'Home of Record' systems (Vista/CPRS) and peripheral Radiology PACS & Dictation Systems.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,No – agency does not have access to source code.,N/A,No,,More than 12 months,Yes,,No,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Implementation of a Severe COVID-19 Risk Prediction Tool,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,"The project seeks to implement a Severe COVID-19 Risk Prediction Tool to effectively identify Veterans at higher risk of developing severe COVID-19 symptoms through a machine-learning model to address one of the key barriers to increased vaccination offerings identified, i.e., the lack of tools for assessing risk of severe disease progression and potential vaccination benefit. It is expected by facilitating the estimated risk for severe COVID-19 progression, as well as a patient's current vaccine eligibility, that these patients can be outreached to and offered the vaccine in a more timely manner to improve vaccination rates and ultimately health outcomes.","The model will output a relative risk score for currently hospitalized patients based on their COVID-19 progression severity risk, and will generate a task notification to clinical teams to notify them of patients within that cohort output who also currently meet eligibility for receiving the COVID vaccine based on CDC guidelines (i.e. no recent history of already receiving vaccine, or recent COVID-19 infection).",Acquisition and/or Development,Both,3/1/2024,3/1/2024,,,Developed with both contracting and in-house resources.,Contract Number: GS-35F-0086U Contract Order Number: 36C10B21F0353,Yes,Veterans Health Administration,,No,,Yes,Yes,Yes,,"VA data input variables for this model were adapted from prior work done at VA Boston through Office of Research / IRB work, and included: Time since infection prior to vaccination Use of glucocorticoids Lymphocyte levels Presence of various comorbidities including: -Alzheimer's Disease -Anemia -Atrial fibrillation -Chronic Kidney Disease -Chronic obstructive pulmonary disease -Epilepsy -Heart failure -Liver disease -Mobility impairments -Pressure and chronic ulcers -Acute myocardial infarction -Hypertension -Lung Cancer -Schizophrenia Tobacco use Body Mass Index categorization -Extreme Obesity -Obesity I / Obesity II -Overweight -Unknown -Underweight Vaccine type (manufacturer) Age group","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ","[""Age"",""Sex/Gender"",""Other""]",Mobility impairment,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Corporate Data Warehouse (VASI ID #1152),6-12 months,No,,No,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, CVI42,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf14/K141480.pdf).,Operation and Maintenance,Both,8/2/2024,8/2/2024,8/2/2024,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Discharge Predictive Model,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,"Implementation of a predictive model that can identify acute inpatients likely to discharge within 24 hours can facilitate communication across clinical and ancillary staff teams who are otherwise currently reliant on a manual process that requires clinical teams to enter a Pre-Discharge order, a process which can be inconsistent due to trainee turnover. The expectation of this project is increased awareness and communication of the hospital's anticipated aggregate discharge volume resulting in potentially improved visibility on hospital throughput for bed coordination, including improved capability and flexibility to accommodate additional hospital admissions and transfers.",The model estimates the overall clinical readiness of inpatient patients using fundamental parameters including vital signs and lab result trends and then produces an output that aggregates the total anticipated number of patients across the facility that would be expected to be ready for discharge within 24 hours based on basic clinical criteria.,Acquisition and/or Development,Neither,6/21/2023,6/21/2023,,,Developed with both contracting and in-house resources.,"GS-35F-0086U, 36C10B21F0353",Yes,Veterans Health Administration,,No,,Yes,Yes,Yes,,"VHA Corporate Data Warehouse for Facility / Station 600 (VA Long Beach Healthcare System). Data types included in Training and Development: • Admission Date • Discharge Date • Laboratory Results (Basic and comprehensive metabolic panel, Complete Blood Count) • Vital Signs (Blood Pressure, Heart Rate, Pulse Oximetry, Temperature, Height, Weight, Pain rating) • Inpatient Specialty Service • Age • Race • Ethnicity • Gender","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.","[""Race/Ethnicity"",""Sex/Gender"",""Age""]",,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,Corporate Data Warehouse (VASI ID #1152),6-12 months,No,,No,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, HSRD IIR 19-069 Optimizing Renin Angiotensin System Blocker Use among Veterans with Kidney Disease,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,"Intended purpose: clinical summarization of narrative clinical notes within the electronic health records among patients with CKD to give providers a summary of relevant terms (side effects, adverse events) noted in those notes (exact phrase matches, pointer to document) along with medication and chronic condition information (not using AI) delivered to primary care providers before a clinic encounter to assist in clinical decision making.","Narrative note that goes into the patient's record with the PCP as an additional signer with a summary of why they qualify for the clinical summarization (CKD), when the ACE/ARB was stopped, documentation of notes and phrases/sentences that are relevant to decision making (exact text from note).",Implementation and Assessment,Neither,6/1/2017,6/1/2018,10/1/2024,,Developed in-house.,,Yes,Veterans Health Administration,,No,not applicable (not disseminating information to the public),Yes,Yes,No,,VHA clinical notes during routine conduct of care.,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.","[""Other""]","none, the tool explicitly looks for side effects from ACE ARB medications or any discussion proximal to a mention of an ACE or ARB class medication.",Yes,"Yes – agency has access to source code, but it is not public.",,Yes,"The primary application that supports this is CirrODS, maintained in VA-TRM, was originally developed for different patient population but has been adapted to this use case, and is a minor application under the VA Innovations Office ATO.",More than 12 months,No,,Other,We have in house computing resources allocated to be able to conduct the AI processing required to sustain the tool. We are not required to request it.,Other,"We obtained the compute require multiple years ago in preparation for needing it for this project and others. Please note that we do not use GPU resources as these clinical summarization tools are not LLMs, as the development of the NLP tools predated the escalation of LLM technologies.",Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",,,,,,,,,,,,,,,,,, CT CoPilot,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf16/K161322.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, "Cranial Navigation, Navigation Software Cranial, Navigation Software Craniofacial, Cranial EM System, Automatic Registration iMRI",Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf22/K223288.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Aquilion ONE (TSX-305A/6) V8.9 With AiCE,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,AI Deep Learning CT Reconstruction,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf18/K183046.pdf).,Implementation and Assessment,Both,5/12/2023,5/12/2023,5/12/2023,,,644-B08013,Yes,Veterans Health Administration,,No,,Yes,No,Yes,," CT scans ","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ","[""Age"",""Sex/Gender"",""Race/Ethnicity"",""Other""]",anatomy,Yes,No – agency does not have access to source code.,,Yes,MD-LITE ID: 2278,More than 12 months,Other,"Controlled by OEM, Canon Medical.",Other,"Controlled by OEM, Canon Medical.",Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, CellaVision DM1200 with the body fluid application,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf10/K102778.pdf).,Operation and Maintenance,Both,8/22/2023,8/22/2023,8/22/2023,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Nediser reports QA,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,"Nediser is a continuously trained artificial intelligence “radiology resident” that assists radiologists in drafting their reports. Using computer vision/ML, Nediser selects normal templates (for example, the knee, shoulder joint), detects absence or presence of hardware in the image, evaluate patella alignment and leg length and angle discrepancy, measure Cobb angles, and evaluate hip exams for osteoarthritis. The output is directed to a QC field in the report that the Radiologist may use. The output helps standardize reports and improve accuracy of reporting.","The system classifies arthritis grade and performs measurements which are output to a QC field in dictation software as a draft that the Radiologist may or may not incorporate into their report templates. This is similar to a Radiology resident who provides draft reports that the Attending Radiologist reviews. The system can also detect objects and landmarks to assist in making measurements and identify abnormality. The output are landmark annotations and measurements. However, these outputs are not sent to the production PACS. They are stored in a secure directory which the Radiologists can refer to and do not become part of the medical record. All original production DICOM images are deleted after processing. Nediser saves de-identified JPG images and reports (no PHI) for audit and review of system performance, and to further train the model is necessary.",Operation and Maintenance,Both,12/1/2020,12/1/2020,1/1/2021,,Developed in-house.,,No,,,No,,No,No,Yes,," Radiology images within VISN 21 are used for training and fine-tuning. The original data set for training the system comes from Stanford University through IRB-approved research. ","Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",,,Yes,"Yes – agency has access to source code, but it is not public.",,No,Nediser,Less than 6 months,No,,No,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,Yes,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.","Nediser is a system that receives Radiology images from the PACS. The system uses AI/computer vision/CNNs to perform several tasks including: identify landmarks on the images to perform measurements of lengths and angles of the lower legs and spine, identifies the absence or presence of hardware, and classifies the grade of hip arthritis. Before the models were deployed in operation, their performance were evaluated with testing and validation, and performance have been published in peer reviewed scientific conferences and/or publications: Ha, AY., Do, BH, Bartret, AL, Fang, CX, Hsiao, A., Lutz, AM, Banerjee, I., Riley, GM., Rubin, DL., Stevens, KJ, Wang, E., Wang, W., Beaulieu, CF., Hurt, B. Automating scoliosis measurements in radiographic studies with machine learning: Comparing artificial intelligence and clinical reports. Journal of Digital Imaging, Feb 2022. https://link.springer.com/article/10.1007/s10278-022-00595-x Larson, N., Nguyen, C., Do, B., Kaul, A., Larson, A., Wang, S., Wang, E., Bultman, E., Stevens, K., Pai, J., Ha, A., Boutin, R., Fredericson, M., Do, L., Fang, C. Artificial Intelligence System for Automatic Quantitative Analysis and Radiology Reporting of Leg Length Radiographs. J Digit Imaging July 2022. https://doi.org/10.1007/s10278-022-00671-2. Ha, A., Fang, C., Baig, S., Do, BH. Automatic Detection of Aortic Aneurysm on CT Exams Using Deep Convolutional Neural Networks. Scientific presentation. AMIA 2020 Annual Symposium, Virtual, November, 2020. Aryan Kaul1, Jason Pai2, Charles Fang2, Ed Boas3, Kathryn Stevens2, Jason Saleh2, Vananh Nguyen2, Constance Chu, Jamie Schroder2, Michelle Nguyen2, Joshua Reicher2, Woon Teck Yap2, Amelie Lutz2, Bao H. Do. Automatic Diagnosis of Knee Patella Malalignment on X-ray Using Artificial Intelligence: Performance Comparison with Physicians. Scientific presentation. AMIA 2020 Annual Symposium, Virtual, November, 2020. Audrey Ha, John Vorhies, Andrew Campion, Charles Fang, Michael Fadell II, Steve Dou, Safwan Halabi, David Larson, Emily Wang, YongJin Lee, Joanna Langner, Japsimran Kaur, Bao Do. Automatic Extraction of Skeletal Maturity from Whole Body Pediatric Scoliosis X-rays Using Regional Proposal and Compound Scaling Convolutional Neural Networks. Scientific presentation, Virtual. IEEE International Conference on Bioinformatics and Biomedicine (BIBM), December 2020. The goal is to deploy this system VISN wide in VISN21. However, the past 3 years have been spent testing in a limited, ""shadow"" deployment. Limited = Palo Alto VA only. ""Shadow"" deployment = the system is never deployed in an autonomous workflow, requires human in the loop, and outputs of the system (annotations of measurements) are never sent back to the PACS or EHRM. Instead, the model's output is sent as text to a marge field/QC field in the Radiologist dictation software called Powerscribe. This does not go to VistA or CPRS or the patient medical record. The merge field is then used by Radiologists if they desire, imported into their draft report. They can review this output, validate visually, and agree or disagree. The program deletes original DICOM data. All annotations of the program, predictions are saved with final radiologist report for audit. Because the AI system requires human in the loop, never sends output to the ERHM/VistA, and is an optional QC/merge field in the report draft, the technical risks are minimized. Clinically, the model performs orthopedic measurements (patella, leg lengths) and arthritis. The models are not involved in real-time decision support of clinical diseases such as stroke, lung cancer. However, all systems have risk. We identify the key risk of the Nediser system is that there may be discrepancy between the system draft output and the final report if the Radiologist does not read the AI draft report and inadvertently saves the report.",TRUE,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,N/A - individuals are not interacting with the AI for this use case,,No,This AI performs computer vision to detect and classify hip arthritis based on the images.,"[""None of the above""]",,No – it is not operationally practical to offer this.,"In Radiology, if patients or providers have a question or concern about a Radiology report, they contact the Radiologist on the report who can discuss and amend the report for errors if necessary.",Other,The final radiology report does not change whether the radiologist uses the AI to perform a measurement with validation by the radiologist vs performing the measurement manually. "CARTO 3 EP Navigation System Software V8.0 (FG-5400-00, FG-5400-00U)",Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf23/K231207.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Care Assessment Needs (CAN) Score,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,CAN is a set of risk-stratifying logistic regression models run on Veterans receiving health care through VHA.,90-day hospitalization or 90-day death,Operation and Maintenance,Both,1/1/2011,6/1/2011,1/1/2012,,Developed in-house.,,Yes,Veterans Health Administration,,No,,No,No,Yes,,"Patient demographics, medical conditions, laboratory results, medications, health care utilization, social determinants of health, social history, vital signs.","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ","[""Race/Ethnicity"",""Sex/Gender"",""Age"",""Socioeconomic Status""]",,Yes,"Yes – agency has access to source code, but it is not public.",,No,,6-12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,Yes,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.",Drift. Monitoring routinely,TRUE,"Automated and Regularly Scheduled Updates: Some aspects of the plan to monitor the AI system post-deployment are automated including re-training of models after detecting drift; however, data science teams are still significantly involved in the monitoring and re-deployment process.",No - Some individual decisions or actions require direct human oversight.,,Online - in the terms or instructions for the service.,,No,Included in pool of candidate variables and model tested for bias,"[""Direct user testing""]",,No – it is not operationally practical to offer this.,The model is for use by physicians as one additional clinical input. Ultimately the physician makes decisions after looking at all patient clinical information including demographics. Physicians can offer feedback but the patient is not directly exposed to this model.,Yes,Physicians can choose to use the model or not and their decision is always with human input. Cartesion Prime (PCD-1000A/3) V10.15,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf23/K231748.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, VA CART Percutaneous Coronary Intervention SYNTAX Score ,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,The VA CART SYNTAX risk model predicts 30-day post-PCI mortality using a combination of clinical and anatomic variables. The model is available to facilitate personalized informed consent discussions and appropriate preparation for high-risk PCI cases.,"The calculated score indicates a risk of major adverse cardiovascular events (death, MI, stroke, and repeat revascularization) over time.",Operation and Maintenance,Both,1/1/2018,3/1/2018,7/2/2019,,Developed in-house.,,Yes,Veterans Health Administration,,No,,No,Yes,Yes,,"coronary anatomic data including, coronary dominance, lesion stenosis, lesion characteristics (ostial, occluded, thrombus, bifurcation, calcification).","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,"coronary anatomic data including, coronary dominance, lesion stenosis, lesion characteristics (ostial, occluded, thrombus, bifurcation, calcification).",Yes,"Yes – agency has access to source code, but it is not public.",,Yes,"Clinical Assessment, Reporting and Tracking Application (VASI ID 1079, EMAS ID 1808) ",6-12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,Yes,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.",Risk: Drift. The model is reassessed periodically.,TRUE,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,Online - in the terms or instructions for the service.,,No,Included in pool of candidate variables and model tested for bias,"[""Direct user testing""]",,No – it is not operationally practical to offer this.,The model is for use by physicians as one additional clinical input. Ultimately the physician makes decisions after looking at all patient clinical information including demographics. Physicians can offer feedback but the patient is not directly exposed to this model.,Yes,Physicians can choose to use the model or not. Clinical decisions are made by the physician not the model. Cardiac CT Function Software Application,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf24/K241038.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Falls Prediction Tool,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Government Services (includes Benefits and Service Delivery),,None of the above.,"Background: Pact Act Co-Pay exemption determination is currently a manual process, and some veterans may not know if they had toxic exposure and may be eligible for co-pay exemption. Objective: Develop an algorithm for the PACT Act to predict if patients are eligible for co-pay exemption. ",Automatically generate a binary output that indicates whether a veteran is eligible for Pact Act Co-Pay exemption.,Acquisition and/or Development,Safety-impacting,5/15/2024,5/20/2024,,,Developed in-house.,,Yes,Veterans Health Administration,,No,,Yes,No,Yes,,Corporate Data Warehouse (CDW); HDAP Self Service Client Portal;,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.","[""Race/Ethnicity"",""Sex/Gender"",""Age""]",I do not have access to the variables at this time,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, VA CART Percutaneous Coronary Intervention Mortality Model ,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,The VA CART mortality risk model is based on logistic regression using baseline clinical and procedural variables to predict post-PCI 30-day mortality. The model is available to facilitate risk stratification at the point of care., Post-PCI 30-day mortality,Operation and Maintenance,Both,1/1/2020,3/1/2021,12/14/2021,,Developed in-house.,,Yes,Veterans Health Administration,,No,,No,Yes,Yes,,"Patient characteristics (demographics, comorbidities and prior procedures, procedural characteristics, labs, and medication history.","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.","[""Age""]",,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,"Clinical Assessment, Reporting and Tracking Application (VASI ID 1079, EMAS ID 1808) ",6-12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,Yes,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.",Risk: Drift. The model is reassessed periodically.,TRUE,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,Online - in the terms or instructions for the service.,,No,Included in pool of candidate variables,"[""Direct user testing""]",,No – it is not operationally practical to offer this.,The model is for use by physicians as one additional clinical input. Ultimately the physician makes decisions after looking at all patient clinical information including demographics. Physicians can offer feedback but the patient is not directly exposed to this model.,Yes,Physicians can choose to use the model or not. Clinical decisions are made by the physician not the model. Pact Act Co-Pay Exemption Prediction Tool,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Government Services (includes Benefits and Service Delivery),,None of the above.,"Background: Pact Act Co-Pay exemption determination is currently a manual process, and some veterans may not know if they had toxic exposure and may be eligible for co-pay exemption. Objective: Develop an algorithm for the PACT Act to predict if patients are eligible for co-pay exemption.",Automatically generate a binary output that indicates whether a veteran is eligible for Pact Act Co-Pay exemption.,Acquisition and/or Development,"Rights-Impacting ",5/15/2024,5/20/2024,,,Developed in-house.,,Yes,Veterans Health Administration,,No,,Yes,No,Yes,,Corporate Data Warehouse (CDW); HDAP Self Service Client Portal;,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.","[""Race/Ethnicity"",""Sex/Gender"",""Age""]",I do not have access to the variables at this time,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",,,,,,,,,,,,,,,,,, VA CART Percutaneous Coronary Intervention Nephropathy Model ,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,The VA CART nephropathy risk model is based on logistic regression using baseline clinical and procedural variables to predict. The model is available to facilitate risk stratification at the point of care.,Post-PCI acute kidney injury,Operation and Maintenance,Both,1/1/2020,6/1/2020,10/1/2022,,Developed in-house.,,Yes,Veterans Health Administration,,No,,No,Yes,Yes,,"Patient characteristics (demographics, comorbidities and prior procedures, procedural characteristics, labs, and medication history.","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.","[""Sex/Gender"",""Age""]",,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,"Clinical Assessment, Reporting and Tracking Application (VASI ID 1079, EMAS ID 1808) ",6-12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,Yes,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.",,TRUE,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,Online - in the terms or instructions for the service.,,No,Included in pool of candidate variables,"[""Direct user testing""]",,No – it is not operationally practical to offer this.,The model is for use by physicians as one additional clinical input. Ultimately the physician makes decisions after looking at all patient clinical information including demographics. Physicians can offer feedback but the patient is not directly exposed to this model.,Yes,Physicians can choose to use the model or not. Clinical decisions are made by the physician not the model. Purchase Order Filing,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Mission-Enabling (internal agency support),,Digitizing text from scanned documents or smart forms for archival purposes using AI.,"We use Microsoft Power Automate's AI builder to file our purchase orders on Sharepoint, as well as documents connected with them (invoices, returns, amendments, packing slips, etc). This saves staff time and makes it easy to respond to audits because all of the required documents are in 1 place with a consistent naming convention.","The AI puts the document into a PDF on Sharepoint and names it according to the Purchase Order Number and document type (538Q407568Amendment1, for example). The AI makes a folder for each PO number and all documents connected to that PO number go in the same folder.",Operation and Maintenance,Neither,5/1/2023,5/18/2023,7/28/2023,,,,,,,,,,,,,,,,,,,,Yes,Microsoft Power Platform -E ID:1780,,,,,,,,,,,,,,,,,,,,,,,,,,, "Brainlab Elements Image Fusion, Contouring (4.5);Image Fusion (4.5);Fibertracking (2.0);BOLD MRI Mapping (1.0);Image Fusion Angio (1.0)",Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf22/K223106.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, VA CART Myocardial Ischemia NLP,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,Extracts myocardial ischemia information in VistA using Natural Language Processing (NLP) of Radiology report. ,"Results of the NLP contain 0 indicating negative stress test result, 1 indicating positive stress test result, and -1 indicating inconclusive stress test result.",Operation and Maintenance,Neither,12/1/2021,5/1/2022,10/30/2023,,Developed in-house.,,Yes,Veterans Health Administration,,No,,No,Yes,Yes,,VistA Radiology report impression text,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,"Clinical Assessment, Reporting and Tracking Application (VASI ID 1079, EMAS ID 1808) ",6-12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, VA CART Ejection Fraction NLP,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,Extracts ejection fraction (EF) information in VistA using Natural Language Processing (NLP) of Echocardiogram TIU report text,"Results of the NLP are separated by EF determination method including Human Observation, Biplane, 4-Chamber, and 2-Chamber, with the Best EF Value following that hierarchy in order. A NULL Best EF Value indicates that the LLM did not find an ejection fraction in the text.",Acquisition and/or Development,Neither,12/1/2021,5/1/2022,,,Developed in-house.,,Yes,Veterans Health Administration,,No,,No,Yes,Yes,,VistA Echocardiogram TIU report text,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,"Yes – agency has access to source code, but it is not public.",,Yes,"Clinical Assessment, Reporting and Tracking Application (VASI ID 1079, EMAS ID 1808) ",6-12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, BioPlex 2200 ANA Screen with Medical Decision Support Software for Use with BioPlex 2200 Multi-Analyte Detection System,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf4/K043341.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, VA CART Percutaneous Coronary Intervention Bleeding Risk Model,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,The VA CART bleeding risk model is based on logistic regression using baseline clinical and procedural variables to predict post-PCI in-hospital bleeding events. The model is available to facilitate risk stratification at the point of care.,Post Percutaneous Coronary Intervention bleed prior to hospital discharge,Operation and Maintenance,Both,1/1/2020,6/5/2020,7/1/2021,,Developed in-house.,,Yes,Veterans Health Administration,,No,,No,Yes,Yes,,"Patient characteristics (demographics, comorbidities and prior procedures, procedural characteristics, labs, medication history, and Post-PCI bleed prior to hospital discharge.","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.","[""Other""]","Height, Weight, clinical characteristics: PCI status, indications, cardiogenic shock, PAD, AFIB, CKD, lab values: hemoglobin, INR, Troponin",Yes,"Yes – agency has access to source code, but it is not public.",,Yes,"Clinical Assessment, Reporting and Tracking Application (VASI ID 1079, EMAS ID 1808) ",6-12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Documentation has been published: Complete documentation has been published to a repository or data catalog within the agency and is made accessible to other data science teams for review and feedback.,No – Agency did not request an extension for this use case.,Yes,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.",Risk: Drift. The model is reassessed periodically.,TRUE,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,Online - in the terms or instructions for the service.,,No,Included in pool of candidate variables,"[""Direct user testing""]",,No – it is not operationally practical to offer this.,The model is for use by physicians as one additional clinical input. Ultimately the physician makes decisions after looking at all patient clinical information including demographics. Physicians can offer feedback but the patient is not directly exposed to this model.,Yes,Physicians can choose to use the model or not and their decision is always with human input. "Biograph Vision, Biograph MCT Family Of PET/CTs",Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf19/K193248.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Rythm Express,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,Predict potassium levels from ECG.,Potassium levels from ECG evaluation.,Acquisition and/or Development,Neither,9/26/2024,9/26/2024,,,Developed with contracting resources.,,Yes,Veterans Health Administration,,No,,Yes,No,No,,,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,"Yes – agency has access to source code, but it is not public.",,No,,Less than 6 months,No,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, AutoContour Model RADAC V3,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf23/K230685.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, A computer vision framework (CVF) for the image classification in the Veteran Health Administration (VHA),Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,"Intended purpose: To develop a computer vision framework (CVF) to classify medical test images in the VHA. Benefits: Disease related decision generation from the medical test image.","The CVF can provide a decision for the given image (e.g., the patient has a disease or not).",Acquisition and/or Development,Safety-impacting,7/15/2024,8/1/2024,,,Developed in-house.,,Yes,Veterans Health Administration,,No,N/A,No,No,No,,Here is the link: https://data.mendeley.com/datasets/rscbjbr9sj/2,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,N/A,Yes,"Yes – agency has access to source code, but it is not public.",N/A,No,N/A,Less than 6 months,Yes,,Yes,,No,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Github Copilot,Department of Veterans Affairs,VA,OIT: Office of Information & Technology,Mission-Enabling (internal agency support),,Collaborating in real-time using AI-assisted tools in word processors.,"The intended purpose of GitHub Copilot is to assist developers by providing intelligent code suggestions and completions, thereby enhancing their productivity and efficiency. Increased Productivity: By suggesting code snippets and completions, Copilot reduces the amount of time developers spend writing boilerplate code and looking up syntax or API usage. Improved Code Quality: Copilot can suggest best practices and common patterns, helping developers write more robust and maintainable code. Learning Aid: It serves as a learning tool for developers by providing examples and explanations of how to implement certain functionalities. Reduced Cognitive Load: By handling repetitive coding tasks, Copilot allows developers to focus more on the logic and design of their applications. Faster Prototyping: Developers can quickly generate code for new features or prototypes, accelerating the development process. Overall, GitHub Copilot aims to streamline the coding process, making it faster and more efficient for developers to build high-quality software.","The AI system's outputs in this context would be: Code Suggestions: The AI might suggest alternative ways to securely handle sensitive information like the master_key. For example, it could recommend using environment variables or secret management services instead of hardcoding the key. Security Warnings: The AI could provide warnings about potential security risks, such as the exposure of sensitive information in the code. Best Practices: The AI might suggest best practices for managing configuration settings, such as using a configuration file or a dedicated secrets management tool. Documentation: The AI could generate comments or documentation strings explaining the purpose of the master_key and how it should be used securely.",Initiated,Neither,1/30/2023,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, aPROMISE X,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf22/K220590.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, AI coding,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Mission-Enabling (internal agency support),,Creating visual representations of data sets for reports and presentations using AI.,"Increased Productivity: By automating routine tasks, both coders and designers can work faster and more efficiently. Improved Quality: AI can enhance the quality of code and designs through intelligent analysis and suggestions. Accessibility: It can make coding and design more accessible to non-experts, democratizing creative processes. Innovation: By offering new perspectives and solutions, AI can drive innovation in both fields. Overall, the integration of AI into coding and creative design aims to enhance human capabilities, foster creativity, and improve overall outcomes in both areas.","Code Snippets: Automatically generated blocks of code based on user prompts or requirements. Bug Reports: Detailed analyses of code with identified issues and suggestions for fixes. Documentation: Automatically generated documentation for codebases, explaining functions, classes, and usage. Refactored Code: Improved versions of existing code that enhance readability, efficiency, or performance. Test Cases: Suggested test cases or automated tests generated to ensure code functionality and reliability.",Initiated,Neither,8/1/2024,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, "Aplio i900, Aplio i800 and Aplio i700 Software V8.1 Diagnostic Ultrasound System",Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf23/K233195.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, "Alignment System Cranial, Alignment Software Cranial, Cirq Alignment Software Cranial Biopsy, Cirq Alignment Software Cranial sEEG, Varioguide Alignment Software Cranial",Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf22/K223864.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Dental - Dentsply Sirona CERE,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,Creating visual representations of data sets for reports and presentations using AI.,Intelligent automation enables a fast proposed dental reconstruction proposal based on the scan. ,Dental reconstruction proposal.,Operation and Maintenance,Both,,,,,,,,,,,,,,,,,,,,,,,Yes,MD-LITE ID: 2278,,,,,,,,,,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Fusion 2024 (JAWS/ZoomText),Department of Veterans Affairs,VA,OIT: Office of Information & Technology,Mission-Enabling (internal agency support),,Searching for information using AI.,"Allows blind and low vision employees to gain information from charts, graphs, tables and other images on the screen to provide comparable access in support of legal requirements of Section 508.","Detailed descriptions and details of charts, graphs, tables and other images on the screen. It also allows for follow on questions to obtain additional information such as trends, specific details, etc.",Operation and Maintenance,Neither,5/2/2024,5/2/2024,6/25/2024,,,,,,,,,,,,,,,,,,,,Yes,Section 508 Tools ID: 1333,,,,,,,,,,,,,,,,,,,,,,,,,,, "Acumen Hypotension Prediction Index - EV1000 Clinical Platform, Acumen Hypotension Prediction Index - Hemosphere Advanced Monitoring Platform, Acumen Hypotension Prediction Index, Hemosphere Advanced Monitoring Platform - Pressure",Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Health & Medical,,None of the above.,FDA-cleared medical device to assist clinicians and improve healthcare outcomes.,Described in FDA documentation for this device (https://www.accessdata.fda.gov/cdrh_docs/pdf18/K183646.pdf).,Operation and Maintenance,Both,,,,,Developed with contracting resources.,,No,,,No,,No,No,Other,Not needed for this device.,N/A,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,Other,Not applicable for this third party device.,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,Yes – Agency requested an extension for this use case.,,,,,,,,,,,,,,,,, Xtract WDS - OSSO,Department of Veterans Affairs,VA,VHA: Veterans Health Administration,Law & Justice,,Creating visual representations of data sets for reports and presentations using AI.,Heat map detection for weapon detection software.,Areas of interest for a heat map that include objects that would produce heat indicators aligned with heat signatures of known types of different weapons.,Acquisition and/or Development,Both,1/26/2023,1/26/2023,,,Developed with contracting resources.,,No,,,No,,No,No,No,,N/A,"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,No,No – agency does not have access to source code.,,No,,Less than 6 months,Other,N/A,Other,N/A,Other,N/A,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,,,,,,,,,,,,,,,,,, Transcription Services,Department of Veterans Affairs ,VA,OIG,Mission-Enabling,,Transcribing and summarizing a recorded meeting or interview using AI.,Increased efficiency,transcribed meetings and interviews,Operation and Maintenance,Neither,7/1/2024,7/1/2024,7/1/2024,,Developed in-house.,,No,,,No,,Yes,No,,,,,,,Yes,Yes – source code is publicly available.,Https://www.noopensourcedotherusingAzureservices.com,No,,Less than 6 months,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,Planned or in-progress.,"Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.",,,,No - Some individual decisions or actions require direct human oversight.,,N/A - individuals are not interacting with the AI for this use case,,No,,,,,,Yes, Chatbot within ESMS,Department of Veterans Affairs ,VA,OIG,Mission-Enabling,,Searching for information using AI.,Increased efficiency,"internal facing, chatbot to assist with policy questions",Acquisition and/or Development,Neither,11/25/2024,12/2/2024,,,Developed with contracting resources.,,No,,,No,,No,No,No,,Current policies and SOPs,,,,No,No – agency does not have access to source code.,,No,Ivanti,,Yes,,No,,,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,,No – Agency did not request an extension for this use case.,Planned or in-progress.,,,,,No - Some individual decisions or actions require direct human oversight.,,N/A - individuals are not interacting with the AI for this use case,,No,,,,,,Yes, Extreme Load Analysis,Tennessee Valley Authority,TVA,Enteprise Planning,Energy & the Environment,,Creating visual representations of data sets for reports and presentations using AI.,Extreme load analysis involves estimating the likelihood of peaks in energy demand for different months of the year and hours of the day. This will aid in energy forecasting,"An RShiny Web App displays EVD for different months of the year, hours of the day, and plants. The Web App allows users to filter for different time periods and different plants. Users can also adjust percentiles to see different quantile values.",Operation and Maintenance,Neither,1/30/2024,1/30/2024,5/1/2024,,Developed in-house.,,No,,,No,Data quality and model quality checks,No,,Yes,,"Weather Data: Going back to 1960. ELRF Data: Going to back to 1980.","Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,Yes,,Yes,R Shiny Web application,6-12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.",No – Agency did not request an extension for this use case.,Yes,Yes,"Model accuracy, drift",,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,None of the above,,No,N/A - no demographic information used,N/A - no demographic information used,,Yes,,Yes, Materials Intelligent Catalog Assistant,Tennessee Valley Authority,TVA,Supply Chain ,Energy & the Environment,,Searching for information using AI.,Transform how inventory is viewed in catalog. Provide intelligent search capability to quickly find what user needs. Reduce duplication of inventory ordering due to inventory and components,"Intelligent Search engine with filters to search through inventory. Also provides information on item, location, cost, and identifies like or similar items",Operation and Maintenance,Neither,12/1/2019,12/1/2019,9/30/2020,,Developed in-house.,,No,,,No,Data quality and model quality checks,No,,Yes,,Maximo data,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,Yes,Yes,,Yes,MS Azure ,More than 12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,Yes,Yes,"Model accuracy, data quality",,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,None of the above,,No,N/A - no demographic information used,N/A - no demographic information used,,Yes,,Yes, Vegetation Management,Tennessee Valley Authority,TVA,Right of Way,Energy & the Environment,,Creating visual representations of data sets for reports and presentations using AI.,Monitoring of vegetation around transmission line and tower and predictive analytics on when/where to trim. Using NV5 to generate the reports,Report on areas identified where vegetation needs to be trimmed,Operation and Maintenance,Neither,,,,,Developed with contracting resources.,,No,,,No,Data quality and model quality checks,No,,Yes,,image data,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,No,No – agency does not have access to source code.,,No,,,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,Yes,Yes,,,"Established Process of Machine Learning Operations: Alongside automated testing and drift detection, model re-training and re-deployments are supported by continuous integration pipelines that are managed by machine learning and data engineers on the platform, adapting work done by data science team into repeatable scripts for re-training and re-testing a model once deployed.",No - Some individual decisions or actions require direct human oversight.,,None of the above,,No,N/A - no demographic information used,N/A - no demographic information used,,Yes,,Yes, CAP Automation,Tennessee Valley Authority,TVA,Enterprise Analytics & Innovation,Energy & the Environment,,None of the above.,"Use NLP to classify condition reports as C, E, or WO level","A classification label - C, E, or WO",Operation and Maintenance,Neither,5/19/2023,5/19/2023,4/24/2024,,Developed in-house.,,No,,,No,Data quality and model quality checks,No,,Yes,,Condition reports,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,Yes,Yes,,Yes,Python application,More than 12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,Yes,Yes,Model accuracy. ,,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,None of the above,,No,N/A - no demographic information used,N/A - no demographic information used,,Yes,,Yes, CR intelligent search,Tennessee Valley Authority,TVA,Enterprise Analytics & Innovation,Energy & the Environment,,Searching for information using AI.,Use NLP to search through CRs. Also tool to include pattern recognition and adding additional safety data to search for and do topic modeling on,Search engine for looking through condition reports,Operation and Maintenance,Neither,6/15/2021,6/15/2021,3/31/2022,,Developed in-house.,,No,,,No,Data quality and model quality checks,No,,Yes,,Condition reports,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,Yes,Yes,,Yes,Python Dash,6-12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,Yes,Yes,,,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,None of the above,,No,N/A - no demographic information used,N/A - no demographic information used,,Yes,,Yes, Materials tracking,Tennessee Valley Authority,TVA,Supply Chain ,Energy & the Environment,,Creating visual representations of data sets for reports and presentations using AI.,Use predictive analytics and confidence levels to track materials as they go through the supply chain process and risk on being delivered on time,"Dashboard for monitoring where in the supply chain process, item is and if it is at risk for being late",Operation and Maintenance,Neither,1/5/2020,1/5/2020,5/1/2021,,Developed in-house.,,No,,,No,Data quality and model quality checks,No,,Yes,,Maximo data,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,Yes,,Yes,"MS Azure, PowerBI",More than 12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,Yes,Yes,,,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,None of the above,,No,N/A - no demographic information used,N/A - no demographic information used,,Yes,,Yes, Heliviewer,Tennessee Valley Authority,TVA,Enterprise Analytics & Innovation,Energy & the Environment,,Searching for information using AI.,Use of helicopter image data for survey and project insights,"Search engine of helicopter data to search through images, projects, flight patterns, etc.",Operation and Maintenance,Neither,1/22/2022,1/22/2022,7/20/2022,,Developed in-house.,,No,,,No,Data quality and model quality checks,No,,Yes,,Helicopter images,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,Yes,Yes,,Yes,MS Azure,More than 12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,Yes,Yes,,,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,None of the above,,No,N/A - no demographic information used,N/A - no demographic information used,,Yes,,Yes, ICI camera ,Tennessee Valley Authority,TVA,Enterprise Analytics & Innovation,Energy & the Environment,,None of the above.,Use computer vision to detect rate of change of water temperature with thermal image data which is indicative of dam leakage,Predictions on what temperature of water should be and risk flag if over threshold,Operation and Maintenance,Neither,11/5/2020,11/5/2020,8/15/2021,10/15/2021,Developed in-house.,,No,,,No,Data quality and model quality checks,No,,Yes,,Thermal camera imagery,"Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.",,,Yes,Yes,,Yes,R Shiny Web application,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,Yes,Yes,,,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,N/A - individuals are not interacting with the AI for this use case,,No,N/A - no demographic information used,N/A - no demographic information used,,Yes,,Yes, GET,Tennessee Valley Authority,TVA,Generation Projects & Fleet Strategy,Energy & the Environment,,Creating visual representations of data sets for reports and presentations using AI.,"Groundwater evaluation Tool - The GET Tool is designed to provide statistical analysis of groundwater monitoring data required under State regulatory programs and the federal Coal Combustion Residuals (CCR) rule. It uses state-of-the-art statistical and predictive methods to evaluate geochemical and geotechnical data under three phases of groundwater monitoring: Detection, Assessment, and Corrective Action. In “Detection”, background data is used to construct prediction limits for use in identifying statistically significant changes from background levels. In “Assessment” and “Corrective Action”, compliance data is used to compute confidence intervals and confidence bands to assess violations of groundwater protection standards (GWPS) to inform of corrective action needs.",Dashboard and reports of statistical analysis of groundwater data,Operation and Maintenance,Neither,12/1/2021,12/1/2021,7/21/2022,,Developed in-house.,,No,,,No,Data quality and model quality checks,No,,Yes,,Groundwater data from Equis,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,Yes,Yes,,Yes,R Shiny Web application,More than 12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,Yes,Yes,,,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,None of the above,,No,N/A - no demographic information used,N/A - no demographic information used,,Yes,,Yes, GMET,Tennessee Valley Authority,TVA,Generation Projects & Fleet Strategy,Energy & the Environment,,Creating visual representations of data sets for reports and presentations using AI.,The Groundwater Modeling Evaluation Tool (GMET) is an R Shiny application that uses Phreeqc geochemical models and analysis to evaluate groundwater chemistry data at each of TVA’s CCR sites ,Geochemical reports on chemical makeup of groundwater data,Operation and Maintenance,Neither,7/23/2023,7/23/2023,2/6/2024,,Developed in-house.,,No,,,No,Data quality and model quality checks,No,,Yes,,Groundwater data from Equis,"Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,Yes,Yes,,Yes,R Shiny Web application,6-12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,Yes,Yes,,,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,None of the above,,No,N/A - no demographic information used,N/A - no demographic information used,,Yes,,Yes, Optiwatt EV Load Model,Tennessee Valley Authority,TVA,Commercial Energy Solutions,Energy & the Environment,,None of the above.,Linear model to predict hourly unitized charging load from non-fleet EVs in a typical meteorological year (TMY).,Hourly load shape profiles for Evs,Operation and Maintenance,Neither,1/22/2024,1/22/2024,2/23/2024,,Developed in-house.,,No,,,No,Data quality and model quality checks,No,,Yes,,"Optiwatt charging data, ELRF weather, TMY Weather","Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,Yes,,Yes,R Shiny Web application,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,Yes,Yes,,,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,None of the above,,No,N/A - no demographic information used,N/A - no demographic information used,,Yes,,Yes, Cold-Weather Heat Pump Model,Tennessee Valley Authority,TVA,Commercial Energy Solutions,Energy & the Environment,,None of the above.,KNN regression to predict hourly unitized impact of an upgraded CWHP in a residential home under historical and forecasted weather conditions.,Hourly load shape profile for heat pump,Operation and Maintenance,Neither,7/19/2023,7/26/2023,9/7/2023,,Developed in-house.,,No,,,No,Data quality and model quality checks,No,,Yes,,"TVA TRM, ELRF, TMY Weather","Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,Yes,,Yes,R Shiny Web application,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,Yes,Yes,,,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,None of the above,,No,N/A - no demographic information used,N/A - no demographic information used,,Yes,,Yes, EE/DR Smart Thermostat Impact Model,Tennessee Valley Authority,TVA,Commercial Energy Solutions,Energy & the Environment,,None of the above.,DR dispatch simulation to demonstrate hourly unitized impact of an EE/DR smart thermostat in a residential home under historical and forecasted weather & load conditions.,Hourly load shape profile for EEDR,Operation and Maintenance,Neither,6/7/2023,6/7/2023,9/29/2023,,Developed in-house.,,No,,,No,Data quality and model quality checks,No,,Yes,,"TVA TRM, TVA Smart Thermostat Pilot data, ELRF, TMY Weather","Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,Yes,,Yes,R Shiny Web application,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,Yes,Yes,,,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,None of the above,,No,N/A - no demographic information used,N/A - no demographic information used,,Yes,,Yes, Standard EE/BE Measure Impact Models,Tennessee Valley Authority,TVA,Commercial Energy Solutions,Energy & the Environment,,None of the above.,Set of ~200 linear models for other EE & BE measures to predict hourly unitized impactunder historical and forecasted weather conditions. Based on a standardized set of calendar and weather features.,Hourly load shape profiles for EE/BE,Operation and Maintenance,Neither,8/1/2019,8/1/2019,10/1/2019,,Developed in-house.,,No,,,No,Data quality and model quality checks,No,,Yes,,"TVA TRM, ELRF","Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,Yes,,Yes,R Shiny Web application,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,Yes,Yes,,,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,None of the above,,No,N/A - no demographic information used,N/A - no demographic information used,,Yes,,Yes, EnergyRightPSPImpactsForecasting,Tennessee Valley Authority,TVA,Commercial Energy Solutions,Energy & the Environment,,None of the above.,"This app is (1) an ETL tool for ancillary data (2) a light-weight MLOps tool for the modeling of energy efficiency and electrification measure-level impact shapes, primarily as it relates to system planning assumptions and (3) a user interface to ES&P's forecast aggregation engine as a supply-side resource.",Generation of EEDR shapes and forecasts,Operation and Maintenance,Neither,8/1/2019,8/1/2019,12/1/2021,,Developed in-house.,,No,,,No,Data quality and model quality checks,No,,Yes,,"EnergyRight and Advanced Analytics databases (components originally from Enterprise Planning system marginal value analysis, system load forecast, TVA EE/BE technical resource manual).","Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,Yes,,Yes,R Shiny Web application,Less than 6 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,Yes,Yes,,,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,None of the above,,No,N/A - no demographic information used,N/A - no demographic information used,,Yes,,Yes, CESASolarModel,Tennessee Valley Authority,TVA,Commercial Energy Solutions,Energy & the Environment,,None of the above.,"This app allows internal TVA users to generate sample bills for directly served and BCD-rate customers based on active wholesale rates, historical load, and expected shape/magnitude of new behind-the-meter solar arrays.",Sample bills based on input data,Operation and Maintenance,Neither,3/3/2021,5/19/2021,1/7/2022,,Developed in-house.,,No,,,Yes,Data quality and model quality checks,No,,Yes,,"Power Billing (PBS), Electricity Sales Statistics (ESS), EIA Henry Hub gas prices and weather, Enterprise Planning solar shape library, PV Watts solar shape generator.","Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,Yes,,Yes,R Shiny Web application,More than 12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,Yes,Yes,,,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,None of the above,,No,N/A - no demographic information used,N/A - no demographic information used,,Yes,,Yes, CESAFCAAllocator,Tennessee Valley Authority,TVA,Commercial Energy Solutions,Energy & the Environment,,None of the above.,"This app is used by one of two co-calculators for the 3-class allocation of TVA's total monthly fuel cost (TMFC) to large general service customers, large manufacturing customers, and customers smaller than 5 MW.",Monthly fuel cost,Operation and Maintenance,Neither,3/31/2021,3/31/2021,1/25/2022,,Developed in-house.,,No,,,Yes,Data quality and model quality checks,No,,Yes,,Electricity Sales Statistics (ESS) via CESA data warehouse (CESADB),"Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,Yes,,Yes,R Shiny Web application,6-12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,Yes,Yes,,,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,None of the above,,No,N/A - no demographic information used,N/A - no demographic information used,,Yes,,Yes, EnergyRightDERPortfolioEvaluation,Tennessee Valley Authority,TVA,Commercial Energy Solutions,Energy & the Environment,,None of the above.,This app is primarily a visualization and export tool for the outputs of ES&P's system planning processes through which Enterprise Planning can obtain aggregations of ES&P forecasts and ES&P can gain insight into evaluation methodologies. It also contains a connection to the ES&P forecast aggregation engine for reporting purposes and stochastic simulation tools.,Report of ES&Ps aggregated forecasts,Operation and Maintenance,Neither,1/1/2019,1/1/2019,11/1/2023,,Developed in-house.,,No,,,Yes,Data quality and model quality checks,No,,Yes,,"EnergyRight and Advanced Analytics databases (components originally from Enterprise Planning system marginal value analysis, system load forecast, TVA EE/BE technical resource manual).","Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.",,,Yes,Yes,,Yes,R Shiny Web application,6-12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,Yes,Yes,,,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,None of the above,,No,N/A - no demographic information used,N/A - no demographic information used,,Yes,,Yes, Prism Models for MDM,Tennessee Valley Authority,TVA,Monitoring Diagnostics Management,Energy & the Environment,,None of the above.,Use of clustering and nearest neighbor models to analyze analog signal data looking for variations in signals. Goal is to detect signal anomalies and provide alarms on deviations.,OMR - measure of how far current vector is from normal . Output includes an automated generated emails for human in the loop validation,Operation and Maintenance,Neither,5/1/2013,5/1/2013,10/1/2013,,Developed with both contracting and in-house resources.,,No,,,,Data quality and model quality checks,,,Yes,,,,,,,,,Yes,Prism,More than 12 months,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.,No – Agency did not request an extension for this use case.,Yes,Yes,,,"Established Process of Machine Learning Operations: Alongside automated testing and drift detection, model re-training and re-deployments are supported by continuous integration pipelines that are managed by machine learning and data engineers on the platform, adapting work done by data science team into repeatable scripts for re-training and re-testing a model once deployed.",No - Some individual decisions or actions require direct human oversight.,,Email,,No,N/A - no demographic information used,N/A - no demographic information used,,Yes,,Yes, Load Forecasting,Tennessee Valley Authority,TVA,Enterprise Planning,Energy & the Environment,,None of the above.,Predict energy and demand out to 30 years in order to support power supply planning,30 year capacity and energy forecasts,Operation and Maintenance,Neither,9/1/2010,10/13/2010,5/12/2011,,Developed with both contracting and in-house resources.,,No,,,,Data quality and model quality checks,No,,Yes,,"Economic forecasts, EEDR forecasts, EV forecasts, commodity forecasts","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,Yes,Itron system,,Yes,,Yes,,Yes,,Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.,"Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.",No – Agency did not request an extension for this use case.,Yes,Yes,Data quality,,"Established Process of Machine Learning Operations: Alongside automated testing and drift detection, model re-training and re-deployments are supported by continuous integration pipelines that are managed by machine learning and data engineers on the platform, adapting work done by data science team into repeatable scripts for re-training and re-testing a model once deployed.",No - Some individual decisions or actions require direct human oversight.,,None of the above,,No,N/A - no demographic information used,N/A - no demographic information used,,Yes,,Yes, ArcGIS,Tennessee Valley Authority,TVA,GIS,Energy & the Environment,,Creating visual representations of data sets for reports and presentations using AI.,"Platform for mapping, analyzing, managing, and viewing geospatial data. It is designed to support and integrate with a variety of GIS services, providing a robust solution for organizations to leverage spatial information.","Maps, spatial analysis, image processing and image analysis",Operation and Maintenance,Neither,4/17/2023,6/10/2023,7/27/2023,,Developed with both contracting and in-house resources.,,No,,,,Data quality and model quality checks,No,,Yes,,"GIS data, ESRI, image data, Maximo data","Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. ",,,No,No – agency does not have access to source code.,,Yes,ArcGIS ESRI,,Yes,,Yes,,Yes,,None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.,,No – Agency did not request an extension for this use case.,Yes,Yes,Data quality,,"Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. ",No - Some individual decisions or actions require direct human oversight.,,None of the above,,No,N/A - no demographic information used,N/A - no demographic information used,,Yes,,Yes, Advanced Network Anomaly Detection,Tennessee Valley Authority,TVA,Network,Mission-Enabling,,None of the above.,Threat hunting and Security Operations Center (SOC) analysts leverage automated tooling to refine alerts that enable discovery and detection of anomalous behavior across the network. These tools use a massive amount of data and provide intelligence beyond what could be seen from a single vector. Correlated network traffic data with proxy communications help identify compromised systems and networks.,"Metrics, Alerting, and optional preventative/proactive protections",Operation and Maintenance,Neither,9/23/2021,9/23/2021,9/23/2021,,,PO 7030410,No,,,No,,No,Yes,,Commercial-off-the-shelf,,,,,No,No – agency does not have access to source code.,,Yes,Corporate,,Yes,,No,,Yes,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Security Information and Event Management (SIEM) Alerting Models,Tennessee Valley Authority,TVA,Cybersecurity,Mission-Enabling,,None of the above.,"Leveraged for comprehensive context interpretation, rapid event detection, and greater productivity with human-assisted automation to SecOps, ITOps. Enables faster alerting, review, and response to terabytes of log data. These automated tools support further refining of alerts based on aggregated data and analyst expertise.","Metrics, Alerting, and optional preventative/proactive protections",Operation and Maintenance,Neither,9/9/2011,9/9/2011,1/3/2019,,,PO 309131,No,,,No,,No,Yes,,Commercial-off-the-shelf,,,,,No,No – agency does not have access to source code.,,Yes,Corporate,,Yes,,No,,Yes,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Vulnerability Tracking,Tennessee Valley Authority,TVA,Cybersecurity,Mission-Enabling,,None of the above.,"Vulnerability analysts track 1000s of vulnerabilities in real time across and enterprise of technologies. Prioritizing severity is a challenging task when there can be 100s of the same level of severity CVE. The tools in use help prioritize data based on contextual information within the environment and industry while also feeding other factors such as exploitability,","Metrics, Alerting, and optional preventative/proactive protections",Initiated,Neither,2/22/2024,2/22/2024,2/22/2024,,,PO 7596319,No,,,No,,No,Yes,,Commercial-off-the-shelf,,,,,No,No – agency does not have access to source code.,,Yes,Corporate,,Yes,,No,,Yes,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Advanced Threat Protection,Tennessee Valley Authority,TVA,Cybersecurity,Mission-Enabling,,None of the above.,"Protecting against attackers and exploitations means testing using the same strategies that adversaries use. Leveraging billions of data points across operating systems, mail flow, and identity patterns, mathematical and probabilistic based decisions can be made to mitigate and even completed remove threats to the organization. ML and AI technology help reduce phishing emails, malware spread, leveraging compromised accounts, and further promote visibility to threats.","Metrics, Alerting, and optional preventative/proactive protections",Operation and Maintenance,Neither,9/30/2016,9/30/2016,9/30/2016,,,Contract 11861,No,,,No,,No,Yes,,Commercial-off-the-shelf,,,,,No,No – agency does not have access to source code.,,Yes,Corporate,,Yes,,No,,Yes,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, AI -Driven Adaptive Protection,Tennessee Valley Authority,TVA,Cybersecurity,Mission-Enabling,,None of the above.,"Enables the real-time exchange of machine-readable cyber threat indicators and defensive measures to help protect against and ultimately reduce the prevalence of cyber incidents. This is useful in helping protect networks against human-operated ransomware, where a threat actor can quickly adjust and maneuver inside the network. Leveraging predictive analytics, the technology can actively engage blocking measures to stop further exploitation.","Metrics, Alerting, and optional preventative/proactive protections",Operation and Maintenance,Neither,9/30/2016,9/30/2016,9/30/2016,,,Contract 11861,No,,,No,,No,Yes,,Commercial-off-the-shelf,,,,,No,No – agency does not have access to source code.,,Yes,Corporate,,Yes,,No,,Yes,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Attack Surface Management,Tennessee Valley Authority,TVA,Cybersecurity,Mission-Enabling,,None of the above.,"Proprietary machine learning models help discover the systems and applications belonging to TVA, across the entire internet. This enables TVA to inventory and take action to ensure protections are in place for external services and applications. These ML tools continuously monitor the attack surface and help prioritize remediation actions for external TVA assets. ","Metrics, Alerting, and optional preventative/proactive protections",Operation and Maintenance,Neither,,,,,,TBD,No,,,No,,No,Yes,,Commercial-off-the-shelf,,,,,No,No – agency does not have access to source code.,,Yes,Corporate,,Yes,,No,,Yes,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Extended Threat Detection,Tennessee Valley Authority,TVA,Cybersecurity,Mission-Enabling,,None of the above.,"Detecting threats within any environment requires large amounts of data, baselines of normalcy, and ability to detect anomalous behaviors within that baseline. Automated detection capabilities leveraging deep packet inspection and real-time analysis to identify the patterns that deviate from what's considered normal enables threat detection and response. Use of unsupervised machine learning models, that learn from previously seen behaviors and are able to determine when something becomes malicious or when an event is considered benign is paramount. New activities are closely monitored in order to most accurately attribute to benign or malicious. Overall, this AI based technology vastly improves visibility and accuracy in threat detection. ","Metrics, Alerting, and optional preventative/proactive protections",Operation and Maintenance,Neither,1/10/2018,1/10/2018,1/30/2019,,,PO 3496587,No,,,No,,No,Yes,,Commercial-off-the-shelf,,,,,No,No – agency does not have access to source code.,,Yes,Corporate,,Yes,,No,,Yes,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Augmented Email Protection,Tennessee Valley Authority,TVA,Cybersecurity,Mission-Enabling,,None of the above.,Email protection service that analyzes activity of messages within email services to determine if messages and associated content embedded within is malicious based on signals that include Al-generated signals regarding normative human behavior and known malicious content. ,"Metrics, Alerting, and optional preventative/proactive protections",Operation and Maintenance,Neither,4/23/2024,4/23/2024,4/23/2024,,,PO 7621572,No,,,No,,No,Yes,,Commercial-off-the-shelf,,,,,No,No – agency does not have access to source code.,,Yes,Corporate,,Yes,,No,,Yes,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Natural Reader,Tennessee Valley Authority,TVA,Workforce Development,Mission-Enabling,,None of the above.,"Text-to-speech desktop software for individual use. This easy-to-use software with natural-sounding voices can read any text such as Microsoft Word files, webpages, PDF files, and E-mails. ",Speech-to-Text,Operation and Maintenance,Neither,7/28/2023,7/28/2023,7/28/2023,,,PO 7502095,,,,,,,,,,,,,,,,,,Corporate,,Yes,,No,,Yes,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, River Forecast,Tennessee Valley Authority,TVA,Riverfleet Management Services,Mission-Enabling,,None of the above.,A multi-scale graph neural network-based autoregressive model. It is trained on historical weather data from ECMWF's ERA5 reanalysis archive. The tool generates predictions at 6-hour time steps for a set of surface and atmospheric variables.,Atomospheric variable predictions,Initiated,Neither,6/24/2024,,,,,,,,,,,,,,,,,,,,,,,Corporate,,Yes,,No,,Yes,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Security Operations Enhancement,Tennessee Valley Authority,TVA,Cybersecurity,Mission-Enabling,,None of the above.,"Empowering SOC analysts with AI-enriched threat intelligence to understand landscapes, security posturing, and live attacks. Promoting the prioritization of event handling and investigative actions to identify and stop attacks by optimizing decision-making.","Metrics, Alerting, and optional preventative/proactive protections",Operation and Maintenance,Neither,9/29/2021,9/29/2021,9/29/2021,,,PO 7040159,No,,,No,,No,Yes,,Commercial-off-the-shelf,,,,,No,No – agency does not have access to source code.,,Yes,Corporate,,Yes,,No,,Yes,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Advanced Threat Protection,Tennessee Valley Authority,TVA,Cybersecurity,Mission-Enabling,,None of the above.,"Protecting against attackers and exploitations means testing using the same strategies that adversaries use. Leveraging billions of data points across operating systems, mail flow, and identity patterns, mathematical and probabilistic based decisions can be made to mitigate and even completed remove threats to the organization. ML and AI technology help reduce phishing emails, malware spread, leveraging compromised accounts, and further promote visibility to threats.","Metrics, Alerting, and optional preventative/proactive protections",Operation and Maintenance,Neither,3/31/2015,3/31/2015,3/31/2015,,,PO 816550,No,,,No,,No,Yes,,Commercial-off-the-shelf,,,,,No,No – agency does not have access to source code.,,Yes,Corporate,,Yes,,No,,Yes,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Perimeter Defense,Tennessee Valley Authority,TVA,Network,Mission-Enabling,,None of the above.,"Using a combination of ruleset definitions specifically designed to detect malicious activity and augmented by a proprietary machine learning algorithm, this tool protects web applications regardless of location. Models trained based on millions of sampled network traffic enables this tool to proactively detect and prevent malicious network activity from reaching and affecting web applications.","Metrics, Alerting, and optional preventative/proactive protections",Acquisition and/or Development,Neither,9/3/2021,9/3/2021,9/3/2021,,,PO 6997377,No,,,No,,No,Yes,Other,Commercial-off-the-shelf,,,,,No,No – agency does not have access to source code.,,Yes,Corporate,,Yes,,No,,Yes,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Intrusion Detection Analysis,Tennessee Valley Authority,TVA,Cybersecurity,Mission-Enabling,,None of the above.,Threat hunting and Security Operations Center (SOC) analysts leverage automated tooling to refine alerts that enable discovery and detection of anomalous behavior across the network. These tools use a massive amount of data and provide intelligence beyond what could be seen from a single vector. Correlated network traffic data with proxy communications help identify compromised systems and networks.,"Metrics, Alerting, and optional preventative/proactive protections",Operation and Maintenance,Neither,9/11/2018,9/11/2018,9/11/2018,,,PO 4314465,No,,,No,,No,Yes,,Commercial-off-the-shelf,,,,,No,No – agency does not have access to source code.,,Yes,Corporate,,Yes,,No,,Yes,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Malware Reverse Engineering,Tennessee Valley Authority,TVA,Cybersecurity,Mission-Enabling,,None of the above.,"Reverse engineering of malware, and software analysis more broadly, will continue to be a critical activity in support of TVA's cyber protections efforts. Analysis of files, URLs, Hashes, etc., to determine if contents are malicious using a combination of different anti-virus scanners and blocking services to advance efficiency and reduce the time spent by analysts analyzing various content within TVA's technology eco-system. ",Threat Intelligence,Operation and Maintenance,Neither,10/26/2018,10/26/2018,10/31/2023,,,PO 4480922,No,,,No,,No,Yes,,Commercial-off-the-shelf,,,,,No,No – agency does not have access to source code.,,Yes,Corporate,,Yes,,No,,Yes,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Threat Intelligence Feed,Tennessee Valley Authority,TVA,Cybersecurity,Mission-Enabling,,None of the above.,Information Sharing; advanced technology and industry expertise to provide participants with near real-time delivery of relevant and actionable threat information of external network traffic.,Threat Intelligence,Operation and Maintenance,Neither,10/7/2020,10/7/2020,10/7/2020,,,N/A,No,,,No,,No,Yes,,Commercial-off-the-shelf,,,,,No,No – agency does not have access to source code.,,Yes,Corporate,,Yes,,No,,Yes,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, AI-Enhanced Search,Tennessee Valley Authority,TVA,T&I Operations,Mission-Enabling,,Searching for information using AI.,"Portal users can find answers more quickly and easily with features like auto-complete search queries, natural language support, and typo handling.",Knowledge Base Catalog Data,Operation and Maintenance,Neither,3/13/2024,6/1/2024,6/1/2024,,,,No,,,No,,Yes,Yes,,Commercial-off-the-shelf,Knowledge Base Catalog Items,"Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.",,,No,No – agency does not have access to source code.,,Yes,Corporate,,Yes,,No,,Yes,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,, Network Anomaly Detection,Tennessee Valley Authority,TVA,Transmission,Mission-Enabling,,None of the above.,"Real-time visibility into network communications, assets, connections, and protocols. ",Detection of anomalous behavior that might impact security or reliability of OT systems,Operation and Maintenance,Neither,6/1/2024,9/1/2024,9/1/2024,,,,No,,,No,,No,No,,Commercial-off-the-shelf,,,,,No,No – agency does not have access to source code.,,Yes,Corporate,,Yes,,No,,Yes,,,,No – Agency did not request an extension for this use case.,,,,,,,,,,,,,,,,,