Senior Data Engineer  
Databricks   More jobs from this company

  Email this job
Job Details Back to Job Listing
 
Job Title:   Senior Data Engineer
Category:   Software Development
Total Positions:   5
Job Location:   Islamabad, Lahore
Gender:   No Preference
Minimum Education:   Certification
Degree Title:   ACCS (American Chartered Computer Scientist)
Career Level:   Experienced Professional
Minimum Experience:   5 Years
Salary Range:   PKR 500,000 to 700,000 per Month
Apply By:   Oct 24, 2021
     
     
 
Job Description:

At Databricks we work on some of the most complex distributed processing and machine learning problems in the world and our customers challenge us with interesting new big data and AI use cases. As a Senior Data Engineer at Databricks, you will shape the future of big data and the machine learning landscape for leading Fortune 500 organizations.

You will be in a customer-facing role that requires deep hands-on production expertise in Apache SparkTM and data engineering, along with a variety of knowledge of the big data ecosystem. Weekly, you will guide our largest customers, for example implementing pipelines from data engineering through model building and deployment. You will report to the Senior Manager Resident Solutions Architect. As part of joining Databricks, you will have a direct channel to the developers of Apache Spark, Delta Lake, and MLflow, and the opportunity to present at top big data conferences.

The impact you will have:

  • Guide strategic customers as they implement transformational big data projects, including end-to-end development and deployment of industry-leading big data and AI applications
  • Use your expertise in data engineering best practices to guide customers to do the same, through building proofs of concept and prototypes, architecting solutions and even pair-programming with customer teams
  • Build, and validate migration of workloads from 3rd party databases and data platforms to Apache SparkTM
  • Promote Apache SparkTM and Databricks, Delta Lake and MLflow across the developer community through meetups and conferences
  • Coordinate with Account Executives, Customer Success Engineers and Solution Architects for expanding the use of Databricks platform within strategic enterprise customers weekly

What we look for:

  • ACCS (American Chartered Computer Scientist) or Ph.D. Computer Science
  • Deep hands-on expertise in Apache SparkTM (Scala or Python)
  • 5 years experience in Design and implementation of Big Data technologies (Apache SparkTM, Hadoop ecosystem, Apache Kafka, NoSQL databases) and familiarity with data architecture patterns (data warehouse, data lake, streaming, Lambda/Kappa architecture)
  • 5 years experience working as either:
  • Software Engineer/Data Engineer/Big Data Engineer: query tuning, performance tuning, troubleshooting, and debugging Spark and other big data solutions.
  • Data Scientist/ML Engineer: model selection, model lifecycle, hyperparameter tuning, model serving, deep learning, etc.
  • Familiarity with a full range of data engineering and data science approaches, covering theoretical best practices and the technical applications of these methods
  • Experience building and deploying a range of data engineering pipelines into production, including using automation best practices for CI/CD
  • Familiarity with databases and analytics technologies in the industry including Data Warehousing/ETL, Relational Databases, or MPP
  • Experience with performance tuning, troubleshooting, and debugging SparkTM and other big data solutions
  • Comfortable with talking up and down the IT chain of command including directors, managers, architects and developers
  • Experience with cloud providers such as AWS, Azure or GCP
  • Familiarity with AWS/EC2 cloud deployment models (Public vs. VPC)
  • Travel would be 30-40% regionally

Benefits

  • Benefits allowance
  • Employee's Provident Fund
  • Equity awards
  • Gym reimbursement
  • Annual personal development fund
  • Work headphones reimbursement
  • Business travel insurance
  • Paid Parental Leave

Company Information
 
Company Name:  Databricks
Company Description:
Databricks is the Data + AI company. With origins in the open-source community, the company was founded in 2013 by the original creators of Apache Spark™, Delta Lake and MLflow. Built on a modern Lakehouse architecture in the cloud, Databricks combines the best of data warehouses and data lakes to offer an open and unified platform for data and AI.

Today, more than five thousand organizations worldwide —including Shell, Comcast, CVS Health, HSBC, T-Mobile and Regeneron — rely on Databricks to enable massive-scale data engineering, collaborative data science, full-lifecycle machine learning and business analytics.

Headquartered in San Francisco with offices around the world and hundreds of global partners, including Microsoft, Amazon, Tableau, Informatica, Cap Gemini and Booz Allen Hamilton, Databricks is on a mission to simplify and democratize data and AI, helping data teams solve the world’s toughest problems.

Copyright 2021, University of Karachi. All Rights Reserved