Big News About the High Performance Fast Computing Challenge

Hello Community,

First off, thanks for your continued patience. We've been overwhelmed by the excitement for the challenge following press coverage and the exceptional community that's come together as a result. This challenge has really taken on a life of its own!

All that being said, we're very aware many of you are eagerly awaiting access to FUN3D software, and still wondering about the status of your application. We wanted to share with you that while it is an important protection, export controlled software is difficult to access, especially when a large number of people are interested in getting access simultaneously (as is the situation with this competition.)

In regard to this situation, it's important to be aware that NASA is committed to making sure the playing field is level and fair for all. And great news, NASA is hard at work creating a custom software package for release through their software release catalog, so you can begin working on your solution that much faster!

The public release of this software is anticipated for the end of June. As you might expect, we're extending the submission deadline by roughly 3 months to allow all competitors ample time to formulate their submissions.

You can expect additional follow-up announcements over the coming weeks with updated instructions and timelines for this new plan.

Thanks so much for sticking with it! We truly couldn't do this without the community commitment and understanding. Please let us know if you have any questions or concerns, and feel free to leave them as a comment to this update.

comments

Matt Craw June 7, 2017, 11:34 p.m. PDT

So I suppose I should not worry about my software request being denied? I mean if a special package will be built, it sounds like I don't have to worry about my denied application. Correct?

reply
- Paul Musille June 8, 2017, 5:52 a.m. PDT
  
  Hi Matt,
  That is correct. For the purposes of this competition the software will be available for download separately from the application process.
  
  reply
Paul Pukite June 6, 2017, 8:42 a.m. PDT

Will be submitting geophysics model of El Nino Southern Oscillation. One of the biggest challenges of climate science.

reply
- Jay McInvale June 6, 2017, 11 a.m. PDT
  
  I suspect NASA will want to use a previously proven model with a bench-marked test case in a relevant subject matter area.
  
  reply
- Paul Pukite June 6, 2017, 11:59 a.m. PDT
  
  I don't really think anyone has made any progress using CFD with respect to significant climate behaviors such as ENSO and QBO. None of the simulations have actually worked to capture anything close to the behaviors observed. I am starting from scratch using ideas from Laplace and Rayleigh and then building off of that. This is more like the concept of Geophysical Fluid Dynamics as espoused by Vallis, see http://rspa.royalsocietypublishing.org/content/472/2192/20160140
  
  reply
Strike Raid June 6, 2017, 7:15 a.m. PDT

The partial release is intriguing, but I’m just wondering how much attention we should be giving to it. If it’s not the critical parts, will the speed increases in those sections really contribute much to the final result. I would hate to go nuts over something that isn’t really all that critical. Will there be any additional information included in the final release. Like, which sections are the most processor intensive, or simply NASA stating, ‘do these first, we really need those’. If this thing runs for days on a supercomputer, I myself would never be able to tell which routines are dragging it down using my Win7 desktop. All the details available would be appreciated.

reply
- Paul Musille June 8, 2017, 6:02 a.m. PDT
  
  Hi Clay,
  There will be documentation released alongside the software. This will both describe the software package as well as identify bottlenecks. Hopefully this will be of great value when getting started!
  
  reply
Seung Name June 6, 2017, 3:11 a.m. PDT

oh well, it's a little sad that this kind of opportunity is only open for US citizens(of course, completely understandable..but still..)

reply
Jay McInvale June 5, 2017, 7:01 p.m. PDT

Once we receive the FUN3D software, must and may I use my own Intel FORTRAN compiler on my Windows 10 workstation?
I understand we are being asked solely to modify that portion of FUN3D software that is used for solving nonlinear partial differential equations written in FORTRAN. I understand our challenge is to speed up the mathematical calculations, and no other functions of the program (like IO).
Can you give us an idea of how much code there is in FUN3D that is subject to being modified in this Challenge? 5000 LinesOfCode, 10000, 25000, 50000...? How many different modules (sub-routines, sub-programs, procedures, etc) are involved? In other words, how much code do we need to look at and hopefully improve within the given 3 months?
What compiler optimizations, directives, and options are allowed? What will and will not work the same on my platform as on the Pleiades supercomputer? Is there any plan to address these types of questions?

reply
- Strike Raid June 6, 2017, 7:07 a.m. PDT
  
  I had read somewhere that FUN3D is 800,000 loc. Can't be sure its an accurate statement, but I did read that.
  
  reply
- Jay McInvale June 6, 2017, 9:51 a.m. PDT
  
  Your comment above is well-taken Clay. It seems silly to think we could do much more then visually inspect the code and offer changes where it seems obvious the code is inefficient. Perhaps compiling and executing is not realistic. It sure seems that way and I thank you for your prescience. We must know how to mark up the code such that NASA can make the changes and test them. However, for NASA to do that for each and every suggested change is unrealistic. It is clear they will need an analyst who can review the suggested changes and pick the winners. They can test the changes only if they have an input deck that models a sufficiently simple test case. I want a full and complete explanation of the procedure that will be used to identify the winners.
  
  reply
- Jay McInvale June 6, 2017, 11:02 a.m. PDT
  
  I am referring to Clay's comment dated June 6, 2017, 7:15 a.m. PDT
  
  reply
- Paul Musille June 8, 2017, 5:55 a.m. PDT
  
  Hi Jay,
  In addition to the software itself, there will be documentation available describing both the software package and identified bottlenecks.
  
  reply
Jack Ebersole June 5, 2017, 1:48 p.m. PDT

So just to be absolutely certain, does that mean the deadline has been pushed to three months from today (September 5) or three months from the original deadline of June 29 (September 29)?

Either way, thank you so much, that really takes a lot of the pressure off.

reply
- Paul Musille June 5, 2017, 3:26 p.m. PDT
  
  Thanks Jack. It's generalities at the moment, so sorry for not being able to share the exact dates just yet, but ~3 months of time with the software. So yes, around the end of September is likely.
  
  reply
James Guthrie June 5, 2017, 12:24 p.m. PDT

Does public release mean the challenge is going to be opened to international participants?

reply
- Paul Musille June 5, 2017, 12:30 p.m. PDT
  
  Hi James, although the process for accessing the software needed to compete will be less cumbersome, the challenge will remain open to only US persons.
  
  reply
- James Guthrie June 5, 2017, 12:32 p.m. PDT
  
  Okay. That's disappointing.
  
  reply
George Delic June 5, 2017, 12:14 p.m. PDT

Before fully answering all the questions asked for in the application the code needs to be reviewed and a profile of the whole package performed. Hence the early release and sufficient delay in closing submissions is essential.

reply
- Paul Musille June 5, 2017, 12:34 p.m. PDT
  
  Precisely George, that is certainly the intent behind the extended timeline: to give competitors time to review the software package and analyze.
  
  reply
- George Delic June 5, 2017, 3:57 p.m. PDT
  
  Paul - thank you for the prompt response. One more request on the release: two input data sets (1) small case for debugging, (2) real-world case input data for performance analysis. Both with corresponding output data for arithmetic validation metrics.
  
  reply
Zach Nwabudike June 5, 2017, 12:08 p.m. PDT

Speaking of community, why not just open source the whole thing?

reply
- Steven Paligo June 5, 2017, 12:16 p.m. PDT
  
  U.S. law limits what code can be openly distributed (see http://www.wrc.noaa.gov/wrso/security_guide/export.htm). There must be parts that are restricted.
  
  reply

Big News About the High Performance Fast Computing Challenge

comments

Thank You!