DevOps Engineer Job at Zenith services, San Antonio, TX

ZC9RaldWdklnWUcyNFN6S0dnREk2T2VmZXc9PQ==
  • Zenith services
  • San Antonio, TX

Job Description

DevOps Engineer - IBM/USAA
Location: San Antonio, TX

Tools & Technologies:

• Apache Kafka (Self-managed or MSK)

• AWS Managed Apache Flink

• Amazon EC2, S3, RDS, and VPC

• Terraform/CloudFormation

• Docker, Kubernetes (EKS)

• ELK Stack, CloudWatch

• Python, Bash

Skills & Expertise:

1.   AWS Managed Services:

◦ Proficiency in AWS services like Amazon MSK, Kinesis, Lambda, S3, EC2, RDS, VPC, and IAM.

◦ Experience with infrastructure management using Terraform or AWS CloudFormation.

2.   Apache Flink:

◦ Understanding of real-time stream and batch data processing with Apache Flink.

◦ Familiarity with Flink's integration with Kafka and managing Flink clusters on AWS.

3.   Kafka Broker (Apache Kafka):

◦ Strong knowledge of Kafka architecture, brokers, topics, partitions, producers, and consumers.

◦ Experience with Kafka management, monitoring, scaling, and optimization, particularly with Amazon MSK or self-managed Kafka clusters.

4.   DevOps & Automation:

◦ Expertise in automating deployments and infrastructure provisioning.

◦ Familiarity with CI/CD pipelines using tools like Jenkins, GitLab, GitHub Actions, CircleCI.

◦ Experience with Docker and Kubernetes for containerizing and orchestrating applications in the cloud.

5.   Programming & Scripting:

◦ Strong scripting skills in Python, Bash, or Go for automation tasks and data pipeline integrations.

6.   Monitoring & Performance Tuning:

◦ Proficiency with monitoring tools like CloudWatch, Prometheus, Grafana.

◦ Expertise in optimizing data pipelines for scalability, fault tolerance, and performance.

Responsibilities:

1.   Infrastructure Design & Implementation:

◦ Design and deploy scalable, fault-tolerant real-time data processing pipelines using Apache Flink and Kafka on AWS.

2.   Platform Management:

◦ Manage and optimize Kafka clusters (MSK or self-managed) and Flink jobs on AWS infrastructure for low-latency, high-throughput processing.

3.   Automation & CI/CD:

◦ Automate infrastructure provisioning and deployment using Terraform, CloudFormation, or other tools.

◦ Integrate new applications into CI/CD pipelines for real-time data processing.

4.   Collaboration with Data Engineering Teams:

◦ Work with Data Engineers, Data Scientists, and DevOps teams to ensure smooth integration and high performance of real-time data systems.

5.   Security and Compliance:

◦ Implement security for Kafka and Flink clusters, ensuring compliance with regulatory standards (e.g., GDPR, HIPAA).

6.   Optimization & Troubleshooting:

◦ Optimize Kafka and Flink performance, troubleshoot issues related to message delivery, job failures, or AWS service outages.

Job Tags

Similar Jobs

Popeyes

General Restaurant Manager Job at Popeyes

 ...General Restaurant Manager at Popeyes Are you a highly skilled and motivated individual with a passion...  ...dining experiences? Do you thrive in a fast-paced environment and enjoy leading a...  ...Popeyes! Popeyes is a renowned fast-food restaurant brand that is committed to... 

Achieve Test Prep

Virtual Assistant - Remote Job at Achieve Test Prep

 ...Work for Us?** Founded in 2008 in the U.S., Achieve is a 100% remote company that hires the best candidates from around the world....  ...the role We are seeking a highly organized and reliable virtual assistant to provide exceptional administrative support to our clients.... 

One80 Intermediaries

Yacht Underwriter Job at One80 Intermediaries

 ...Remote : This position is responsible for the critical review of the insured's boating experience and exercising detailed knowledge of...  ...placement services and binding authority for property and casualty, life, travel/accident and health, affinity and administrative... 

Sunrise Senior Living

Medication Nurse LPN Job at Sunrise Senior Living

 ...communication patterns, history, and basic human needs.* Practices routinely good standard care precautions of cleanliness, hygiene, and health .* Audits medication carts.Resident Care:* Notifies RCD of any resident and/or family concerns through the LEAD process.*... 

Placer County

Accountant Job at Placer County

 ...Accountant - I and Accountant - II . Each level in this job series has different minimum...  ...CHARACTERISTICS This is the entry level class in the Accountant series. This...  ...between accounts; reconcile bond statements and bank statements to the general ledger. Prepare...