Loads

This section provides instructions on how to use Transform4J to accomplish data loads.

Loads require three Transform4J constructs: Transformer, TransformerDataSource, and TransformerDataTarget. TransformerDataSource classes retrieve data from a supported data source such as any relational database with a JDBC driver, CSV files, MongoDB, etc. TransformerDataTarget classes write data to any supported data storage media such as any relational database with a JDBC driver, CSV files, MongoDB, etc. Transformer classes move data from the source to the target making any changes desired along the way.

SimpleLoad Utility

Transform4J provides a basic Transformer for data loads called SimpleLoad and we will use it for all our examples. However, if you want to make custom transformations in the load, it's easy to implement your own Transformer.

The SimpleLoad utility supports functions. Functions are classes that compute additional fields based on the load input. See Functions for more information.

The SimpleLoad utility supports filters. Filters are classes that screen out unwanted input records. See Filters for more information.

You can register field name simple field name changes. Sometimes this is useful if the target is a relational database which has pre-defined names that don't match your input. For example, I can change field 'firstName' in the input to 'FIRST_NAME' in the output without creating a custom transformation.

See SimpleLoad javadoc.

Example: Load MongoDB Database from relational database content

This code runs a MongoDB database load in line (i.e. in the same thread)

JdbcDataSource source = new JdbcDataSource(myDataSource, "select * from application_log");
MongoDbDataTarget target = new MongoDbDataTarget(new ServerPortVO[]{new ServerPortVO("localhost", 27017)}
	"myDbName", "myCollectionName", "myUsername", "myPassword");
	
SimpleLoad loader = new SimpleLoad(source, target);
Transform4j.runTransformation(loader);
				

This code runs the same load in a spawned separate thread asynchronously. Instead of manually creating a new thread for the extract, I could have scheduled it with a product like Spring Batch or Quartz.

JdbcDataSource source = new JdbcDataSource(myDataSource, "select * from application_log");
MongoDbDataTarget target = new MongoDbDataTarget(new ServerPortVO[]{new ServerPortVO("localhost", 27017)}
	"myDbName", "myCollectionName", "myUsername", "myPassword");
	
SimpleLoad loader = new SimpleLoad(source, target);
Transform4j.runTransformation(loader);
Runnable loaderRunnable = Transform4j.makeRunnable(extract);
(new Thread(loaderRunnable)).start();