Note: The job is a remote job and is open to candidates in USA. New York Technology Partners is seeking a Big Data Engineer to design, build, and optimize scalable data platforms that power analytics and AI-driven solutions. The role involves developing high-performance data pipelines and ensuring data accessibility and accuracy across cloud environments.
Responsibilities
- Design and maintain scalable ETL/ELT pipelines for large-scale data processing
- Build data ingestion and transformation frameworks for structured and unstructured data
- Support feature engineering and data preparation for analytics and AI/ML initiatives
- Ensure data quality, reliability, and performance throughout the data lifecycle
- Collaborate with data scientists, analysts, and engineering teams to deliver production-ready datasets
- Optimize cloud-based data workflows for scalability, efficiency, and cost
Skills
- Strong experience with Big Data technologies such as Spark, Hadoop, Kafka, or similar
- Hands-on experience with AWS data services including S3, EMR, Glue, Redshift, and Lambda
- Proficiency in Python, Java, or Scala
- Experience building scalable data pipelines, data models, and ingestion frameworks
- Knowledge of feature engineering, workflow orchestration tools (e.g., Airflow), and data governance best practices
- Exposure to AI/ML data platforms is a plus
Company Overview
Company H1B Sponsorship