{"id":5133,"date":"2026-02-24T12:25:24","date_gmt":"2026-02-24T12:25:24","guid":{"rendered":"https:\/\/www.arivonix.ai\/techsupport\/docs\/pyspark-transformation\/"},"modified":"2026-03-02T11:25:45","modified_gmt":"2026-03-02T11:25:45","password":"","slug":"pyspark-transformation","status":"publish","type":"docs","link":"https:\/\/www.arivonix.ai\/documentation\/docs\/pyspark-transformation\/","title":{"rendered":"PySpark Transformation"},"content":{"rendered":"<section class=\"documentation-content\">\n<h1>PySpark Transformation<\/h1>\n<p>Create and execute PySpark transformations to process data using Spark&#8217;s Python API.<\/p>\n<h2>Creating a PySpark Transformation<\/h2>\n<ol>\n<li>Click the <strong>Plus<\/strong> icon and select <strong>PySpark<\/strong> from the menu. <img decoding=\"async\" class=\"img-fluid img-screenshot\" src=\"\/documentation\/tech-image\/Tech\/PySpark_Transformation\/image_7.12.png\/\" alt=\"Relevant visual\" \/><\/li>\n<li>Wait for the system to establish connection to the server. The Notebook interface will appear.<\/li>\n<li>Select the appropriate <strong>Kernel<\/strong> based on your requirements.<\/li>\n<li>Enter the following details:\n<ul>\n<li><strong>Transformation Name<\/strong><\/li>\n<li><strong>Description<\/strong><\/li>\n<\/ul>\n<\/li>\n<li>Enter your PySpark code according to your transformation requirements.<\/li>\n<li>Click the <strong>Save<\/strong> button in the top-left corner to save your transformation. <img decoding=\"async\" class=\"img-fluid img-screenshot\" src=\"\/documentation\/tech-image\/Tech\/PySpark_Transformation\/image_53.44.png\/\" alt=\"Relevant visual\" \/><\/li>\n<li>Execute the code by clicking <strong>Run<\/strong>.<\/li>\n<li>Close the Notebook window to complete the process.<\/li>\n<\/ol>\n<p>The PySpark transformation is now created and ready for use.<\/p>\n<\/section>\n","protected":false},"excerpt":{"rendered":"<p>PySpark Transformation Create and execute PySpark transformations to process data using Spark&#8217;s Python API. Creating a PySpark Transformation Click the Plus icon and select PySpark from the menu. Wait for the system to establish connection to the server. The Notebook interface will appear. Select the appropriate Kernel based on your requirements. Enter the following details: [&hellip;]<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"doc_category":[178],"doc_tag":[],"class_list":["post-5133","docs","type-docs","status-publish","hentry","doc_category-workflow-designer"],"year_month":"2026-04","word_count":103,"total_views":0,"reactions":{"happy":0,"normal":0,"sad":0},"author_info":{"name":"Hari","author_nicename":"hari","author_url":""},"doc_category_info":[{"term_name":"Workflow Designer","term_url":"https:\/\/www.arivonix.ai\/documentation\/docs-category\/workflow-designer\/"}],"doc_tag_info":[],"_links":{"self":[{"href":"https:\/\/www.arivonix.ai\/documentation\/wp-json\/wp\/v2\/docs\/5133","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.arivonix.ai\/documentation\/wp-json\/wp\/v2\/docs"}],"about":[{"href":"https:\/\/www.arivonix.ai\/documentation\/wp-json\/wp\/v2\/types\/docs"}],"author":[{"embeddable":true,"href":"https:\/\/www.arivonix.ai\/documentation\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/www.arivonix.ai\/documentation\/wp-json\/wp\/v2\/comments?post=5133"}],"version-history":[{"count":1,"href":"https:\/\/www.arivonix.ai\/documentation\/wp-json\/wp\/v2\/docs\/5133\/revisions"}],"predecessor-version":[{"id":5313,"href":"https:\/\/www.arivonix.ai\/documentation\/wp-json\/wp\/v2\/docs\/5133\/revisions\/5313"}],"wp:attachment":[{"href":"https:\/\/www.arivonix.ai\/documentation\/wp-json\/wp\/v2\/media?parent=5133"}],"wp:term":[{"taxonomy":"doc_category","embeddable":true,"href":"https:\/\/www.arivonix.ai\/documentation\/wp-json\/wp\/v2\/doc_category?post=5133"},{"taxonomy":"doc_tag","embeddable":true,"href":"https:\/\/www.arivonix.ai\/documentation\/wp-json\/wp\/v2\/doc_tag?post=5133"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}