...
Synapse-ETL-Jobs is Synapse project where all the python scripts are maintained. To create a new job or update an existing job which requires the processing of python script, first we need to do the changes in Synapse-ETL-Jobs and build. Every merge in the project creates a new version of tag (e.g. v1.63.0). Always use the latest tag version.
...
Synapse-Stack-Builder is the project where all of the cloud-formation scripts are maintained. To create ETL glue job and Glue tables we have added the templates(datawarehouse-template.json.vpt) which creates the required resources. Also update the latest version of Synapse-ETL-Jobs tag in datawarehouse-config.json file.
The cloud-formation stack for ETL jobs should be parameterized by database name. So that on dev environment , engineer can work together on same project without coliding with each other. Basically the each stack will create a stack name with parameter name(database name) and then the job name and all the resources tied to the job will contain the database name name in it e.g job trigger, destination path like bucket/databasename/tablename.
...