Data staging management system
Abstract:
Batch job data staging combining synchronous/asynchronous staging. In pre-processing, a stage-in source file, and a target file for stage-out, in permanent storage, are identified using a batch script. From data amounts, time for stage-in/stage-out to/from temporary storage are estimated. Stage-in is based on the time, stage-out being asynchronous, and each asynchronous staging is classified short/long term depending on the time, each staging being recorded in a table. If a source file is modified, incremental staging is added to the table. With a staging list scheduling for batch jobs stage-in is performed, monitoring progress in the table, and resources may be allocated for the jobs nodes without waiting for stage-in to complete. The job generates results in the temporary storage, and using post-processing, stage-out transfers results to the target file in permanent storage.
Public/Granted literature
Information query
Patent Agency Ranking
0/0