This project utilizes faker-pyspark to generate random schema and dataframes to mimic data table snapshots. Using these snapshots to process and apply SCD2 pattern into delta table as the destination. | |
https://github.com/spsoni/pyspark-delta-scd2 | |
pyspark-delta-scd2-0.4.1 | MIT |
download~amd64 ~x86 | pypi |