PySpark Transformation

Create and execute PySpark transformations to process data using Spark’s Python API.

Creating a PySpark Transformation

1. Click the Plus icon and select PySpark from the menu.
2. Wait for the system to establish connection to the server. The Notebook interface will appear.
3. Select the appropriate Kernel based on your requirements.
4. Enter the following details:
   • Transformation Name
   • Description
5. Enter your PySpark code according to your transformation requirements.
6. Click the Save button in the top-left corner to save your transformation.
7. Execute the code by clicking Run.
8. Close the Notebook window to complete the process.

The PySpark transformation is now created and ready for use.