Decision makers rely on the Intelligence Community to provide accurate and relevant forecasts, and IARPA is working to identify methods to maximize the quality of these forecasts. IARPA’s Geopolitical Forecasting (GF) Challenge invites individuals or teams to develop innovative solutions and methods for integrating crowdsourced forecasts and other data into accurate, timely forecasts of geopolitical events. Solvers will be competing against each other and in parallel with a similarly structured research program funded by IARPA. Over the course of approximately seven months, the GF Challenge solvers will be asked to develop solutions capable of producing forecasts to a series of questions like:
In order to answer these questions, solvers will be given access to a continuously updated stream of forecast judgments produced by a crowd of human forecasters and will be allowed to use other data sources to produce their solutions. The challenge presents an opportunity for individuals and teams to earn prizes by creating methods that successfully forecast of a wide variety of geopolitical events, such as political elections, international conflict, disease outbreaks, and macro-economic indicators.
Solvers will be invited to the GF Challenge Platform to compete. Each solver or team will be assigned a unique application programming interface (API) token and login account for the platform. A steady stream of questions (roughly 25 per month) will be released for solvers to produce probabilistic forecasts against (specific requirements will be described in the rules). Questions may be binary (yes/no), multiple choice, or ordered outcome (binned quantity/date). In addition to the questions, solvers will receive access to a continuously updated stream of forecast judgments produced by a crowd of human forecasters. Solvers are encouraged to create solutions that use this stream of human judgments alongside other publicly available data streams and information to create their forecasting solutions.
Scores will be calculated using a metric which incorporates forecast accuracy and confidence. This metric is based on the Brier score, a quadratic measure of the forecaster's closeness to ground truth for the outcome that occurs. For example, an accurate forecast with 90% confidence for an event that occurs will score four times better than a forecast with 80% confidence. The score will be calculated on a daily basis over the duration of the question; those daily accuracy scores will then be averaged. The more accurate and timely the forecast, the better score.
As questions will range in duration, milestone check-ins for questions resolved during a certain period will be used to award progress-based incentives. This means solvers who start after the challenge has launched are still able to compete for some of the prizes. Finalists who wish to compete for monetary prizes will need to provide the solution package for review the GF Challenge Team at the end of the challenge, or as appropriate, if requested. Solutions will need to be documented and all data streams used in prize-winning solutions will need to be outlined and verified by the GF Challenge Team. Please refer to guidelines for a description of scoring.
Solvers will be able to compete on their own or form teams to compete for a total prize purse of $200,000. The prize amounts shown are dependent upon certain criteria and/or participation requirements. Challenge results and solutions will be reviewed at an IARPA forecasting Workshop in Washington, D.C. For additional details and prize requirements, please refer to the guidelines. Prizes will be distributed for the following criteria:
*Bonuses have additional eligibility requirements. Please see the challenge rules for details.
Individuals and teams, 18 years of age or older will be able to compete for cash prizes. Some individuals or organizations may not be eligible for monetary prizes. However, these solvers may, upon IARPA approval, participate in the challenge and be eligible for ranking on the leaderboard. Approval will need to be obtained in advance of participation by emailing: email@example.com.
Click “Follow” to receive challenge updates and follow its progress!
Can you create a method to forecast the future? The Geopolitical Forecasting (GF) Challenge invites Solvers from around the world to develop innovative solutions and methods for integrating crowdsourced forecasts and other data into accurate, timely forecasts on worldwide issues. The challenge presents an opportunity for individuals and teams to earn prizes by creating methods that successfully demonstrate a forecast of a wide variety of geopolitical events, such as political elections, disease outbreaks, and macro-economic indicators.
Existing methods of geopolitical forecasting include human judgment-intensive methods, such as prediction markets, and data-intensive approaches, such as statistical models. GF Challenge Solvers will develop solutions that produce probabilistic forecasts in response to numerous closed-ended forecasting questions that concern specific, objectively verifiable geopolitical events containing timeframes with deadlines and locations. The effort will run in parallel to IARPA’s most current geopolitical forecasting research program-- Hybrid Forecasting Competition (HFC). Challenge Solvers will be competing on a largely overlapping set of Individual Forecasting Problems (IFPs) as HFC research teams, and given access to the same human forecaster data stream. In addition to the provided data stream, Solvers may use other data streams and their own data and models for the challenge (in accordance with applicable laws).
The following three (3) stages are anticipated as part of the GF Challenge:
Solvers will register on the HeroX platform and acknowledge their understanding and intent to follow the rules of the challenge. Following registration, instructions will be sent to access the application programming interface (API) within the Cultivate Labs platform. Solvers will be granted an API token and documentation for interacting with the challenge platform.
Stage 1 registrations will be accepted on a rolling basis, so Solvers may register at any point. However, solvers will be rewarded by accuracy scores that incentivize participation on their ability to forecast on more IFPs. Eligibility for prizes (documented below) have minimum participation requirements for the resolved IFPs included in each prize.
Forecasting questions, known as IFPs, will be released periodically and will be open for a specified period of time. Solvers should check daily for new IFPs. While an IFP is open, Solvers may update their forecasts daily until the IFP closes. Please see the Judging and Scoring section below for the full-time periods associated with the IFP forecasting period. Each Solver will be permitted to submit forecasts using up to 25 different methodological “slots” as described in the Rules section below. Once the IFP closes, the forecasts will be locked and scores for the IFP for each method will be calculated. See the Judging and Scoring section below for further information. Scores will be tracked on the leaderboard within the Cultivate Labs platform.
In addition to the forecast submission activities, Solvers are encouraged to work to improve their forecasting methods throughout the competition.
Throughout the Challenge, there will be three (3) Milestone Check-Ins at which Milestone Prizes will be awarded. The Milestone Prizes will be awarded based on all IFPs that resolve during the Milestone duration. The Milestone schedule is as follows (milestone prizes will be validated within 30 days of the milestone end):
At the end of the challenge, the grand total scores will be confirmed and determined for each of the prize categories. Solvers will need to provide a write-up of their solution to the Government for final review and evaluation. This final review will determine if the solutions are sound and meet the criteria of the challenge. Solvers will provide a short (4 page) explanation for their solution. Before final prize award, IARPA may ask for additional documentation.
Upon conclusion of the Government evaluation of the documentation provided by the Solvers from the Evaluation stage, prizes will be awarded as follows.
1st Place Overall
2nd Place Overall
3rd Place Overall
4th Place Overall
5th Place Overall
$30,000 split amongst all eligible Solvers/teams
Definitions of prize criteria:
$25,000 (5 awards of $5,000 each)
$7,500 (10 awards of $750 each)
$10,000(10 awards of $1,000 each)
Minimum participation in 80% of the total IFPs resolved within the milestone period
Top ten Solvers with the highest score in the Milestone will each get $1,250
|Spring Forecaster||$2,400 (3 awards of $800 each)|
Solvers will need to submit a W-9 tax form, or a W8-BEN form in order to receive payment. Solvers are responsible for payment of all taxes incurred from the acceptance of Prize funds. Solvers will be given 30 days to submit their paperwork upon notification of award. IARPA and Booz Allen are not responsible for lost or stolen prize payments, or incorrect routing and payment information provided by the winning Solvers.
Solvers will retain the Intellectual Property (IP) rights to their solutions. IARPA will use information submitted through the challenge and application process to understand the technology.
Some individuals or organizations may not be eligible for prizes for reasons listed below. However, these Solvers may, upon IARPA approval, participate in the challenge and be eligible for ranking on the leaderboard. Approval will need to be obtained in advance of participation by emailing .
Any teams who forego prizes will be publicly listed on the HeroX GF Challenge page to ensure full transparency of their competition status.
To be eligible to win a prize under this competition, an individual or entity:
Federal grantees may not use federal funds to develop challenge applications unless consistent with the purpose of their grant award. Federal contractors may not use federal funds from a contract to develop challenge applications or to fund efforts in support of a challenge submission.
Employees of IARPA, Booz Allen Hamilton, Cultivate Labs, HeroX, Good Judgment, Inc., and MITRE are not able to compete in the challenge. Companies and contractors who are currently supporting, or who have previously supported, the above (or other) entities in efforts related to the IARPA HFC program are also not able to participate in the challenge. This restriction extends to members of such persons’ immediate families (spouses, children, siblings, parents), and persons living in the same household as such persons, whether or not related.
Federally Funded Research & Development Centers (FFRDCs) and (DoD) University Affiliated Research Centers (UARCs) not mentioned above may be eligible to submit forecasts and receive leaderboard recognition, but are not eligible to win prize dollar awards. In order to compete, an email must be sent to firstname.lastname@example.org with the team name, organization represented, and team member information.
IARPA’s Hybrid Forecasting Competition (HFC) performer team members and HFC Good Judgment Forecasters, as well as members of such persons’ immediate families (spouses, children, siblings, parents), and persons living in the same household as such persons, whether or not related, are not eligible to participate in the competition.
Federal employees, employees of FFRDCs, and employees of UARCs can participate in the challenge as a private citizen, not affiliated with their organization, as long as (a) the Solver is not employed by an organization that is actively involved in the challenge or closely related programs; (b) the Solver has received approval from their organization, if applicable, to participate in their personal capacity (note, it is the Solver’s responsibility to verify this); and (c) Solvers cannot use government or their organizations’ resources, computers, or access to government information in aid of their participation, except for those resources available to all other participants on an equal basis.
Entrants must agree to assume any and all risks and waive claims against the Federal Government and its related entities, except in the case of willful misconduct, for any injury, death, damage, or loss of property, revenue, or profits, whether direct, indirect, or consequential, arising from their participation in a competition, whether the injury, death, damage, or loss arises through negligence or otherwise.
Entrants must also agree to indemnify the Federal Government against third-party claims for damages arising from or related to competition activities. Entrants are not required to obtain liability insurance or demonstrate financial responsibility in order to participate in the competition.
By participating in the competition, each entrant agrees to comply with and abide by these rules and the decisions of IARPA and/or the individual judges, which shall be final and binding in all respects.
By participating in the competition, each entrant agrees to follow all applicable local, state, federal and country of residence laws and regulations.
Companies, universities, individuals, and Teams of Solvers are able to participate in this challenge. Companies / Teams will need to elect a Team Captain, who will be the main point of contact (POC) for communications. Individuals who have registered on the HeroX GF Challenge page as an individual competitor, will not be eligible to participate in a team. All Team Members must meet the General Eligibility Requirements, self-register, and acknowledge these rules through the HeroX GF Challenge page.
Except for residents of Iran, Cuba, North Korea, Crimea Region of Ukraine, Sudan or Syria or other countries prohibited on the U.S. State Department’s State Sponsors of Terrorism list, Solvers residing in any country are able to participate. In addition, Solvers are not eligible to participate if they are on the Specially Designated National list promulgated and amended, from time to time, by the United States Department of the Treasury. It is the responsibility of the Solver to ensure that they are allowed to participate in this challenge and allowed to export their technology solution to the United States.
By participating in the competition, each entrant hereby agrees that:
Without limiting the generality of the foregoing, IARPA and the U.S. Government, Booz Allen Hamilton, Cultivate Labs, HeroX, and MITRE are not responsible for incomplete, misdirected, prize notifications; or for late, lost, inaccessible, network connections, or for issues with Internet Service Providers, websites, or other connections and third-party sites that the entrants choose to use for the competition; or for any technical malfunctions, failures, difficulties, or other errors of any kind or nature.
These rules cannot be modified except by IARPA. All decisions by IARPA regarding adherence to these rules are final. The invalidity or unenforceability of any provision of these rules shall not affect the validity or enforceability of any other provision. In the event that any provision is determined to be invalid or otherwise unenforceable or illegal, these rules shall otherwise remain in effect and shall be construed in accordance with their terms as if the invalid or illegal provision were not contained herein.
By participating in the competition, each Solver represents, warrants, and covenants as follows:
If the Submission includes any third party works (such as third-party content, data sets, algorithms, or open source code), entrant must be able to provide, upon the request of IARPA, documentation of all appropriate licenses and releases for such third-party works. If entrant cannot provide documentation of all required licenses and releases, IARPA reserves the right to disqualify the applicable Submission, or seek to secure the licenses and releases for the benefit of IARPA, and allow the applicable Submission to remain in the Competition. IARPA also reserves all rights with respect to claims based on any damages caused by participant’s failure to obtain such licenses and releases.
Solvers – whether an individual, a team or an entity – will indemnify, defend, and hold IARPA, Booz Allen Hamilton, Cultivate Labs, HeroX and MITRE from and against all third-party claims, actions, or proceedings of any kind and from any and all damages, liabilities, costs, and expenses relating to or arising from entrant’s submission or any breach or alleged breach of any of the representations, warranties, and covenants of entrant hereunder.
IARPA reserves the right to disqualify any submission that IARPA, in its discretion, deems to violate these Rules. IARPA reserves the right to disqualify any Solver that is operating in a manner that disrupts the competition, does not comply with the spirit of the competition, or is found to be manipulating the competition. IARPA also reserves the right to amend these rules throughout the duration of the contest should extenuating circumstances arise.
Appendix (available in the downloadable Challenge Rules Document)