Schulung - IBM KM404G - IBM InfoSphere Advanced DataStage - Parallel Framework v11.5

Course Outline

1: Introduction to the parallel framework architectureDescribe the parallel processing architectureDescribe pipeline and partition parallelismDescribe the role of the configuration fileDesign a job that creates robust test data2: Compiling and executing jobsDescribe the main parts of the configuration fileDescribe the compile process and the OSH that the compilation process generatesDescribe the role and the main parts of the ScoreDescribe the job execution process3: Partitioning and collecting dataUnderstand how partitioning works in the FrameworkViewing partitioners in the ScoreSelecting partitioning algorithmsGenerate sequences of numbers (surrogate keys) in a partitioned, parallel environment4: Sorting dataSort data in the parallel frameworkFind inserted sorts in the ScoreReduce the number of inserted sortsOptimize Fork-Join jobsUse Sort stages to determine the last row in a groupDescribe sort key and partitioner key logic in the parallel framework5: Buffering in parallel jobsDescribe how buffering works in parallel jobsTune buffers in parallel jobsAvoid buffer contentions6: Parallel framework data typesDescribe virtual data setsDescribe schemasDescribe data type mappings and conversionsDescribe how external data is processedHandle nullsWork with complex data7: Reusable componentsCreate a schema fileRead a sequential file using a schemaDescribe Runtime Column Propagation (RCP)Enable and disable RCPCreate and use shared containers8: Balanced OptimizationEnable Balanced Optimization functionality in DesignerDescribe the Balanced Optimization workflowList the different Balanced Optimization options.Push stage processing to a data sourcePush stage processing to a data targetOptimize a job accessing Hadoop HDFS file systemUnderstand the limitations of Balanced Optimizations 

Objective

Please refer to course overview