MUSCATE news : release of Madagascar Sentinel-2 L2A data

Olivier Hagolle, 12 mai 2017

After a good period with very little production incidents, our Muscate facility has had a tough period, which almost halted the production for one week. Until last week, the Muscate ground segment was running on CNES older cluster, and was just using the new cluster’s bay of disks to store the input and output data. The access to the bay was made through the network and slowed the processing. Last week, we moved Muscate to the new cluster, and also moved physically one of the computers we are using : this brought us a few surprises that took more time than expected. Muscate is now running on the new cluster : it has its own reserved computing nodes, and a dedicated facility for the scheduler. The result is that the processing of a L2A product with MAJA now only takes 23 minutes on average, while it used to be 50 minutes before. All the teams who did that migration, handling the development of Muscate, its exploitation, or the cluster facility, deserve acknowledgement !

We still have a remaining bug on Muscate, that we call the « Friday night fever » bug, which often cancelled of all the processing schedule we had submitted for the week-end. This is particularly bad in April and May, in which, us, French workers, benefit from a large number of days off and long week-ends to rest, cook and eat good food, hike the Pyrenees, or defeat far right candidates. The Friday night fever bug usually happened just after the teams leaved office, on Friday, around 22h. We now know what causes it, and last week-end, the production went smoothly, even if not at full speed. The bug should be corrected soon, and then, we will be able to push Muscate to the full production capacity it was designed for.

So thanks to all this work, Muscate is now easily keeping up with the real time processing of many sites (France, Morocco, Spain, Belgium, Senegal, Tunisia, Reunion, Mali-Burkina), and goes on with the processing of new sites, starting from December 2015. Our counter is now above 12000 ! The data can be downloaded here.We just added the Madagascar site to that list, and started processing the Chinese site, which is a large one. Portugal is the next one..

Rechercher