Python Code-Based Transformations

Python code-based Transformations provide the option to modify ingested Events through code before loading them to the Destination. You can add fields, modify or remove them, and even join fields for specific Events.

Hevo provides the following classes for transforming your data:

• Event: The Event class provides methods with which you can manipulate the properties and metadata of the ingested Events.

• TimeUtils: The TimeUtils class provides methods to perform date and time-related operations on the properties of the ingested Events.

• Utils: The Utils class provides methods that allow you to perform operations such as converting a property to an array or checking if a property is a dictionary.

The Python Transformations UI is organized as follows:

• The left pane, called CODE, is where you can write your Python code using methods from the standard Python libraries and or those provided by Hevo. The Build Output section at the bottom displays errors and or warnings if your Transformation script contains any restricted Python modules and or functions.

• The right pane, where you can:

  • Select a random sample Event or select one from a specific Event Type, and then

  • Test the Transformation code on your sample Event.

Hevo runs your Transformation code on each qualifying Event that is ingested by the Pipeline.

The following is an example of a Transformation you can create in Hevo:

def transform(event):
    # Get event name from the event #
    eventName = event.getEventName()

    # Get properties from the event #
    properties = event.getProperties()

    # Add a new field to the event #
    properties['city_name'] = "London"

    return event

The above snippet adds the city_name field to the ingested Event before loading it to the Destination.

Python libraries supported by Hevo

Hevo compiles and runs your Transformation script with Jython version 2.7, which includes all the standard Python libraries. However, you cannot import third-party Python libraries, such as numpy, scipy, and scikit-learn. Hevo also does not support the use of certain Python modules and functions in the Transformation script.

Hevo highlights any unsupported modules or functions in the script as errors with a red wavy line under them. Similarly, it indicates unsafe modules or functions as warnings with a yellow wavy underline. In these cases, when you click the DEPLOY or TEST button, the Build Output section displays the error message(s) and or warning(s). You can also hover over the red or yellow wavy lines in your code to view these messages. You must fix the errors in your code to test or deploy your Transformation.

In the future, Hevo may also restrict the use of Python modules and functions for which it currently displays a warning. Therefore, it is not recommended to use them in your Transformation script.

The following image shows the Build Output section displaying an error message for the sys module, which is unsupported, and a warning for the json module, which is considered unsafe.

Refer to the following articles for the classes and methods provided by Hevo to write your Transformations code, as well as a few example Transformation scripts:

• Transformation Methods in the Event Class
• TimeUtils
• Utils
• Examples of Python Code-based Transformations