Senior/Principal Software Engineer

WeVision LLC seeks Senior/Principal Software Engineer in Irvine, CA.

Duties:
Design, implement, and maintain big data pipelines and cloud infrastructure for data processing and storage.  Telecommuting from anywhere in the U.S. is permitted.

The detailed duties are as follows:

1. Design, develop, and maintain robust big data pipelines utilizing DataBricks, Scala, and Spark. These pipelines efficiently process terabytes of data, ensuring optimal performance and scalability.

2.  Implement Extract, Transform, Load (ETL) jobs using AWS Glue and Airflow, enabling seamless extraction, transformation, and loading of data into diverse data stores. This streamlines data processing workflows.

3. Engineer data ingestion pipelines utilizing AWS Kinesis, Kafka, and Flume. These pipelines are instrumental in collecting data from a wide array of sources, providing a comprehensive view of the information landscape.

4. Play a pivotal role in data modeling and schema design for Hive data stores. This involves structuring data in a way that optimizes storage and retrieval, enhancing the efficiency of data access operations.

5. Develop and maintain cloud infrastructure using Terraform and AWS services, including S3, EC2, EMR, and Lambda. This ensures a robust and reliable foundation for data processing and storage.

Requirements:

·       Master in computer science or EE, or a related field；

·       2 years of big data development experience;

·       Knowledge of DataBricks, Snowflake, Scala, and Spark; Hadoop, AWS Glue and Airflow; AWS Kinesis, Kafka, and Flume; S3, EC2, EMR, and Lambda.  

Salary: $124,800 to $145,000/yr.

Send CV to yolanda.li@wevision.io