What You'll Do
Develop, test, and maintain data pipelines (ingestion, transformation, quality checks) using PySpark/SparkSQL on Databricks.
Build and modify Airflow DAGs (MWAA) for pipeline orchestration.
Write and optimize SQL queries for data transformation and validation.
Support data quality by implementing and monitoring quality checks (Great Expectations or equivalent).
Participate in code reviews — both giving and receiving feedback.
Investigate pipeline failures and data quality issues with guidance from senior engineers.
Write and maintain documentation for datasets and pipelines you build.
Participate in sprint planning, estimation, and retrospectives.
Follow CI/CD, testing, and governance standards defined by the team.
This is a remote position. A remote position does not require job duties be performed within proximity of a Visa office location. Remote positions may be required to be present at a Visa office with scheduled notice. #LI‑Remote
Qualifications
For this role, you must be based in Brazil
Language Skills
Proficiency in English at B2 level or above (Upper-Intermediate)
Basic Qualifications
3+ years of relevant work experience with a Bachelor's or Associate’s Degree OR 5+ years of relevant work experience.
Python for automation and data processing
Data pipeline concepts (batch, ETL/ELT patterns)
Technical Skills
Apache Spark basics (PySpark or SparkSQL)
Databricks (jobs, workflows, cluster management, tuning)
SQL (intermediate–advanced: joins, window functions, CTEs)
Amazon S3 data lake design (partitioning, layout, lifecycle)
Basic cloud concepts (AWS: S3, IAM, CloudWatch)
Data modeling (dimensional / Kimball, medallion layers)
Git/GitHub workflow and code review
Preferred Qualifications
Delta Lake (ACID tables, OPTIMIZE, VACUUM, schema evolution, MERGE)
Apache Airflow / MWAA (DAG design, retries, backfills, SLAs)
CDC patterns (DMS, incremental processing, MERGE upserts)
Data quality concepts (validation, testing)
Terraform basics
BI integration (Superset, dashboarding)
Visa is an EEO Employer
Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability or protected veteran status. Visa will also consider for employment qualified applicants with criminal histories in a manner consistent with EEOC guidelines and applicable local law.
