Namaste!
I'm Devendra Kumar Singh
Data Engineer with 6+ years of experience in cloud data engineering across Azure Databricks and AWS. I specialize in scalable ETL/ELT pipeline development with Python, PySpark, Apache Spark, and Kafka, alongside data warehouse development, workflow orchestration, data modeling, and pipeline performance and cost optimization.
What I Do?
Data Engineer
⚡ Build and optimize scalable data pipelines and automated ETL/ELT workflows for reliable data ingestion, transformation, and delivery
⚡ Develop cloud data engineering solutions on AWS and Azure Databricks, using services including S3, Glue, and EMR
⚡ Use Python, PySpark, Apache Spark, and Kafka for batch pipelines and real-time data processing
⚡ Orchestrate data workflows with dbt, Dagster, Meltano, and Airflow
⚡ Develop data warehouse models in Snowflake and optimize SQL queries, pipeline performance, and cloud data costs
⚡ Apply data modeling, database design, and schema evolution across relational and NoSQL systems
Edge and Cloud computing
⚡ Deployment, Infrastructure as code (IaC), Cloud Backup, Cloud Computing, Data Migration
⚡ Proficient in managing cloud infrastructure on platforms such as AWS, firebase.
⚡ Designed and deployed scalable cloud architectures for high-traffic web applications, resulting in improved performance and cost-efficiency.
⚡ Implemented automated deployment pipelines using tools like AWS CloudFormation
⚡ Implemented security best practices for cloud environments, including IAM policies, encryption, and network security groups
⚡ Collaborated with development teams to optimize cloud-based applications for scalability, reliability, and cost-effectiveness
⚡ Leveraged edge computing for IoT device management, resulting in improved efficiency and reduced bandwidth usage.
⚡ Developed real-time data processing pipelines using edge devices to enhance performance and reliability.
Web Development
⚡ Building responsive websites ReactJS & NodeJS
⚡ Creating application backend in Node, Flask
⚡ Having experience in Domain & Hosting using AWS, Firebase, Netlify, Vercel, Heroku & Hostinger
⚡ Capable of working on both frontend and backend aspects of web development to build end-to-end solutions.
⚡ Experience in integrating frontend and backend components to create seamless user experiences
⚡ Knowledge of MVC (Model-View-Controller) or similar architectural patterns for structuring and organizing web applications.
Database
⚡ Data Warehousing, Database Design, Database Administration, Schema Design, ETL Development
⚡ Proficient in managing relational databases such as MySQL, PostgreSQL, Oracle, or SQL Server
⚡ Experience with NoSQL databases like MongoDB, Cassandra, or Firebase for handling unstructured data.
⚡ Skilled in database installation, configuration, and maintenance to ensure optimal performance and reliability.
⚡ Designed and implemented database schemas, tables, and relationships to meet application requirements
⚡ Proficient in query optimization techniques to enhance database performance and minimize response times.
Tools
⚡ My helping verbs with development