Senior Data Engineer

Job title:

Senior Data Engineer

Company

ITV

Job description

Senior Data EngineerCommercial Technology & ChangeContract | 6 monthsWhite City, London | HybridHiring range: £600 – £700 per dayFor more information please email [email protected]Your work matters to millions.Shaping culture is in the DNA of ITV. So, it’s not surprising that you’ll find us in every home in the UK, our productions are famous all over the world and we’re at the forefront of the digital streaming revolution.When you join us, you enter a fun working environment. With opportunities to learn, to grow and make a real difference. Small enough that your impact’s felt in the business, but big enough that your impact reaches millions of people.Come develop your skills, change TV and the course of your career. Don’t just watch it. Be part of it. Join ITV.Your impact sends ripples.The teamMillions of people use the products we build. We’re customer-focused, informed by data and have a product mindset, using state-of-the-art cloud-first technology. Iterating and evolving daily. In a fast-changing world, we ensure our systems work effectively, and our Technology teams are spearheading the transformation to a digital-first business. There’s never been a more exciting time to join us.As part of the Commercial Technology & Change department, our engineering team builds multiple products using cloud-native technologies, and puts maintaining a strong developer experience at our core.You will join a cross-functional engineering squad, working closely with fellow Data Engineers, Software Engineers, Data Scientists and a Product Manager.The roleWe are currently looking for a Senior Data Engineer with MLOps experience to join a recently kick-started forecasting project.The ideal candidate will possess strong engineering principles, extensive knowledge of data engineering practices and proficiency in productionising machine learning algorithms and maintenance of ML models.This is an exciting opportunity to work on a cutting-edge project that involves the automation of data ingestion, transformation, feature engineering and model inferencing using cloud-centric technologies.The role will entail but not be limited to:

• Data Product Design & Development: Ownership and delivery of the design, development, and maintenance of scalable data pipelines, infrastructure and data products using Apache Airflow, AWS Glue and PySpark. This includes the implementation of efficient and reliable data ingestion, processing, and storage solutions at scale, as well as prototyping new approaches early in the delivery process.
• Business Engagement: Assist and collaborate with the ITV commercial teams and data product owners to define data product requirements and visualisation needs.
• Data Loading & Transformation: Develop and optimise Extract, Transform, Load (ETL/ELT) processes to ensure the smooth flow of data from various sources into the data platform. Perform data cleansing, validation, and enrichment to support accurate and reliable data analysis.
• Data Modelling & Architecture:

• Collaborate with data scientists, analysts, and stakeholders to understand data requirements and design appropriate data models and architecture. Optimise data storage formats and partitioning for efficient data retrieval and analysis
• As required offer support and assistance to the product management team in designing and building out data dashboards and other visualisation / presentation solutions
• Performance Tuning & Optimisation: Monitor and optimise data pipelines and workflows to ensure high performance and reliability. Identify and resolve performance bottlenecks, data quality issues, and data integration challenges.
• MLOps:

• Support the product ionisation efforts of Machine Learning models, ensuring code is production-ready.
• Support the LAG and Model Feature creation process alongside Data Scientists.
• Assist with the version control, release management and deployment of ML Models.
• Integrate monitoring of ML-based assets and components.
• Data Quality: Define the data quality characteristics of your data products and work with the QA team to implement automated quality processes to measure data correctness and report outcomes through our observability infrastructure.
• Data Governance & Security: Implement data governance practices and ensure compliance with ITV data privacy and security regulations. Establish data access controls, encryption mechanisms, and data retention policies. Execute and comply with ITV architecture governance processes.
• Collaboration & Documentation: Work closely with cross-functional teams, including data scientists, analysts, and software engineers, to understand their requirements and provide data engineering support. Document data pipelines, workflows, data presentation implementations and technical specifications for future reference. Communicate complex solutions in a clear and understandable way to both technologists and business stakeholders.
• Continuous Improvement: Stay up-to-date with industry trends and emerging technologies in the data engineering domain. Identify opportunities for process improvement, automation, and innovation to enhance data engineering capabilities.
• Ways Of Working: Participate in our delivery process by proactively: offering to take work during planning sessions; defining technical user stories; updating the team during stand-ups on progress and issues; ticket/card management; managing expectations relating to delivery of work; working with and improving our CI/CD delivery approach

Skills you’ll need (minimum criteria)

• Proven experience in building Spark-based data projects.
• Strong programming skills in Python (PySpark preferred), with an emphasis on writing clean, efficient, and maintainable code.
• Experience in owning, designing and implementing data pipelines ingesting enterprise levels of data volume. In-depth knowledge of data engineering concepts, data integration, and ELT/ETL processes.
• Understanding of machine learning algorithms and their application in forecasting and prediction tasks.
• Experience with AWS services, specifically MWAA (Managed Workflows for Apache Airflow), AWS Glue and AWS Sagemaker.
• Proficiency in working with data in parquet format and optimising data pipelines for performance.
• Familiarity with test-driven development (TDD) and CI/CD practices.
• Strong problem-solving skills and the ability to work independently and collaboratively in a team environment.
• Excellent communication skills, with the ability to convey complex technical concepts to both technical and non-technical stakeholders.

Other things we’re looking for (key criteria)

• Knowledge of the UK broadcast industry & broadcast/OTT advertising market
• Knowledge of digital marketing and advertising industry
• Familiar with data design techniques for classification and regression as well as knowledge of A/B testing, experimental design, and general statistical modelling
• Familiarity with architecture disciplines such as Data Mesh Architecture, Data Architecture, BI Architecture and Enterprise Architecture
• Good understanding of data governance, data privacy, and security principles.
• Knowledge of tools across other data management capabilities – data engineering, data governance, data quality, metadata management and master data management

ITV is for everyone.ITV strongly encourages applications for this role from disabled people. As a Disability Confident Leader, if you meet the minimum criteria for a role and you have declared that you are disabled, we’ll guarantee to take you to the next stage* (minimum criteria above).We’re happy to discuss any support/personalisation you may need during our application and selection process as part of our reasonable adjustments. Drop us a line if you require anything at .about applying with a disability.

• There may be a few exceptions where we are not able to take all eligible candidates to the next stage due to the volume of applications.

Expected salary

£600 – 700 per day

Location

London

Job date

Sun, 11 Aug 2024 07:10:03 GMT

To help us track our recruitment effort, please indicate in your email/cover letter where (hiring-jobs.com) you saw this job posting.