Transform4J

ETL tools currently in the marketplace tie transformation features to user presentation and business intelligence features, thereby greatly increasing the dependencies and complexity of the toolset. While user presentation (e.g. dashboards, etc.) and business intelligence are very useful and valuable, those features have very little to do with the transformation features that make that data available. There's no technical reason why ETL capabilities need to be tied to business intelligence capabilities. Furthermore, ETL tools have uses besides user presentation and business intelligence.

As a result of this artificial tie between ETL capabilities and business intelligence, ETL tools are very extensive products with very large numbers of dependencies and very large learning curves. Most ETL tools require developers to learn custom/proprietary programming languages and development tools for transformations instead of using common tools that most developers are already familiar with. As a consequence of these custom tools, most ETL tools then add to that complexity to make debugging and defect resolution easier. This statement is true for most open source ETL tools as well as they are laden with very extensive dependencies that make it much more difficult and labor intensive to incorporate into the enterprise. Consequently, using formal ETL tools is often cost prohibitive and developers end up writing custom transformation logic for data copies/transformations, data exports, and data imports.

Introducing Transform4J! Transform4J is a Java framework that allows you to very quickly write data copies/transformations, data exports, and data imports directly in Java. No learning new, proprietary programming languages with very primitive debugging and error reporting abilities (which take time developers typically don't have). You have all the customary error reporting capabilities with detailed stack traces and debugging capabilities that you're used to.

Transform4J has very few dependencies. This makes it quick and easy to incorporate Transform4J into your existing applications. Furthermore as Transform4J is all-java,it's easily combined with batch schedulers, such as Spring Batch or Quartz.

Transform4J is distributed with an Apache 2.0 license, which is acceptable to most organizations. In most organizations, developers won't have any bureaucratic hassles adopting it.

Transform4J was conceived (and its predecessor created) as we needed it. We needed ETL capabilities for one of our current clients and the extensive runtime requirements and dependencies for the ETL tools currently in the marketplace made them cost prohibitive and would have given us bureaucratic obstacles with their adoption.

Features

Transform4J is an ETL framework can effectively be used for the following types of activities:

  • Extracting data from one application and writing a copy of that data for use in another application
  • Data conversion efforts for purchased product implementations
  • Loading data into any supported Transform4J target
  • Maintaining data copies in multiple locations
  • Joining data from multiple sources
  • Performing simple transformations such as deriving new values and separating values into component pieces
  • Maintaining data in data warehouses.

Additional information on ETL concepts can be found here.

Data Sources and Targets

Transform4J supports the following data source and target types:

  • Any JDBC data source or target
  • Any CSV format source or target
  • Any Json format source or target
  • MongoDB sources and targets
  • Cassandra sources and targets
  • HBase sources and targets (forthcoming)