Job title:
AWS BigData Developer- Pentaho
Company
Syncreon Consulting
Job description
Company DescriptionWe provide Recruitment and Staffing services to many industries and domain through our innovative and customized solutions and passionate commitment to research. Ability to understand the hiring strategies, availability of talent and compensation benchmarking makes us proud hiring partner for various industries. We work as trusted business partners and always strive to deliver the most value and highest return on investment for our clients. We are highly trained business professionals with strong understanding of clients need. We work closely with the leading staffing trade associations, training, and research organizations to ensure we are knowledgeable of the latest industry trends and technologies.Job Description
- 6-8 years
- Understand requirement from product owners and translate into requirement and scope documents.
- Decide on the best fitment of technologies/services that are available in scope
- Create Solution for data ingestion and data transformation using Hadoop services like Spark, Spark streaming, Hive, etc.
- Create technical design documents to communicate solutions to the team and mentor the team to develop the solution
- Build the solution with Hadoop Services as per design specifications
- Assist team teams to build test cases and support with testing the solution
- Coordinate with Upstream, Downstream and other supporting teams for production implementation
- Provide post-production support for Solutions implemented
- Develop data engineering frameworks in Spark on AWS Data Lake platform? Coordinate with clients, data users and key stakeholders to understand feature requirements needed merge them to create reusable design patterns
- Data onboarding using the developed frameworks
- Understand and make sense of available code in Netezza and Hadoop to design a best way to implement its current features in AWS Data Lake
- Unit test code and aid with QA/SIT/Perf testing
- Migration to production environment
- Candidate should have strong working experience with Hadoop platform.
- Strong hands-on experience on Hive, Spark with Scala.
- In-depth knowledge and extensive experience in building batch workloads on Hadoop.
- Adept in analyzing and refining requirements, consumption query patterns and choosing the right technology fit like RDBMS, data lake and data warehouse.
- Should have knowledge of analytical data modelling on any of the RDBMS platform / Hive
- Should have working experience in Pentaho
- Proven practical experience in migrating RDBMS based data to Hadoop on-prem
- 7 plus years of data experience in Data warehouse and Data Lake platforms
- At least 2+ years of implementation experience in AWS Data Lake, S3, EMR/Glue, Python, AWS RDS, Amazon Redshift, Amazon Lake Formation, Airflow, Data Models, etc.
- MUST have very strong knowledge of Pyspark
- Must understand data Encryption techniques and be able to implement them
- Must have experience in working with Bitbucket, Artifactory, AWS Code Pipeline
- Hands-on experience working with Terra bytes \ peta bytes scale data and millions of transactions per day.
- Skills to develop ETL pipeline using Airflow? Knowledge of Spark streaming or any other streaming jobs
- Ability to deploy code using AWS Code Pipeline and Bit bucket is an added plus? Expert in any of the following programming language: Scala, Java and comfortable with working on Linux platform.
- Knowledge of Python CI/CD pipeline design AWS Cloud Infrastructure for services like S3, Glue, Secrets manager, KMS, Lambda, et
Regards,Mohammed Ilyas,PH – 229-264-4024 or Text – 229-469-1455 or You can share the updated resume at Mohammed@vtekis. comAdditional InformationAll your information will be kept confidential according to EEO guidelines.
Expected salary
Location
Toronto, ON
Job date
Sat, 15 Feb 2025 23:30:39 GMT
To help us track our recruitment effort, please indicate in your email/cover letter where (hiring-jobs.com) you saw this job posting.