Job Title: Azure Data Engineer Azure Data Factory (ADF)
Work Location: Redmond,WA-98052
Job Details:
Must Have Skills:
Python, SQL, (bonus: Scala), Apache Spark, PySpark, Spark SQL, Azure Databricks (notebooks, Jobs/Workflows, clusters), Azure Synapse, Delta Lake, Parquet, ADLS Gen2
Nice to have skills:
ETL/ELT, MERGE/upsert, partitioning, Spark performance tuning, Git, Azure DevOps / CI/CD
Detailed Job Description
Techniques (Data Engineering Practices)
ETL / ELT
Extract data from source systems
Transform/clean/reshape into usable form
Load into target tables (ELT = transform after loading, inside the platform)
MERGE / Upsert
Update existing rows when records already exist
Insert new rows when they don't
Handled in a single atomic operation (core Delta Lake pattern)
Partitioning
Split large tables by a column (e.g., date, source type)
Enables partition pruning - scan only relevant data
Improves query speed and reduces cost
Spark Performance Tuning
Handle data skew and manage shuffles
Caching / persistence of reused datasets
Right-size partitions; use broadcast joins
Cluster sizing and resource optimization Orm
Minimum years of experience
8-10 years
...with the ultimate goal of enabling human life on Mars. Lead Security Officer The Lead Security Officer is an elite security professional... .... ~5+ years of experience in the military, law enforcement, overseas security contracting, or advanced corporate security realm. ~...
...Netherlands, and the United Kingdom. Primary Function Provide EHS service, solutions, programs, best practices and technical... ...facilities and support functions to effectively and efficiently manage compliance assurance and EHSS risk mitigation. This role will report...
...Blockchain Engineer All IT Solutions United States Pennington, New Jersey Workplace Type On-site Employment Type Contract We are currently seeking a qualified Blockchain Engineer to support this engagement. Please review the complete requirements, work...
...RESPONSIBILITIES Obtains comprehensive data pertinent to the patient from the... ...sitting, reaching and keyboard use/data entry. Frequent use of vision and depth perception... ...Academy of Pediatrics (AAP) Work Shift: Night Shift (United States of America) Job...
...Be part of an amazing story. Macys is more than just a store. Were a story. One that... ...Summary The Barista Trainer supports the Starbucks Manager in running an efficient,... ...following all company standards. Full-time colleagues are typically scheduled to work...