About this role
Job title: Data Engineer
About the Role A Data Engineer at ScotiaTech (Scotiabank's technology hub in Bogota) is responsible for building and maintaining efficient, reliable, and automated data pipelines and systems to ensure seamless data flow across the IBDF. The role focuses on data infrastructure availability, performance, and scalability through automation and monitoring tools, and collaborates with cross-functional teams to optimize data operations and troubleshoot incidents.
What You'll Do
- Design and implement highly automated data pipelines.
- Monitor and troubleshoot data flows in real time.
- Implement data-driven DevOps practices for pipelines.
- Manage permissions and access to data in secure environments.
- Optimize infrastructure to handle large volumes of data.
- Document processes for ease of maintenance and scalability.
What We're Looking For
- Experience in building and maintaining data pipelines using tools like Apache Airflow, Spark or BigQuery.
- Strong knowledge of database management systems (SQL, and NoSQL, e.g. PostgreSQL, MySQL, MongoDB).
- Familiarity with automation and monitoring tools to ensure data infrastructure performance (e.g. Cloud Logging, Apache Airflow).
- Knowledge of agile methodologies for project management and delivery.
- Excellent collaboration skills with operational and infrastructure teams.
- Attention to detail to troubleshoot and fix system errors.
- Time management skills to respond to incidents efficiently.
- Proactive mindset to optimize and improve existing processes.
- 2-3 years of experience in data engineering roles focused on pipelines.
- Proven expertise in managing cloud-based data infrastructure (AWS, Azure, GCP).
- Previous experience working with DataOps or DataOps methodologies.
Nice to Have
- Education: Bachelor's degree in computer science, Data Engineering, or a related field.
- Certifications: AWS Certified DevOps Engineer or Google Cloud Professional Data Engineer.
Compensation & Benefits
- ScotiaTech hub offers an inclusive, positive work environment and competitive benefits.