DevOps Engineer - IBM/USAA
Location: San Antonio, TX
Tools & Technologies:
• Apache Kafka (Self-managed or MSK)
• AWS Managed Apache Flink
• Amazon EC2, S3, RDS, and VPC
• Terraform/CloudFormation
• Docker, Kubernetes (EKS)
• ELK Stack, CloudWatch
• Python, Bash
Skills & Expertise:
1. AWS Managed Services:
◦ Proficiency in AWS services like Amazon MSK, Kinesis, Lambda, S3, EC2, RDS, VPC, and IAM.
◦ Experience with infrastructure management using Terraform or AWS CloudFormation.
2. Apache Flink:
◦ Understanding of real-time stream and batch data processing with Apache Flink.
◦ Familiarity with Flink's integration with Kafka and managing Flink clusters on AWS.
3. Kafka Broker (Apache Kafka):
◦ Strong knowledge of Kafka architecture, brokers, topics, partitions, producers, and consumers.
◦ Experience with Kafka management, monitoring, scaling, and optimization, particularly with Amazon MSK or self-managed Kafka clusters.
4. DevOps & Automation:
◦ Expertise in automating deployments and infrastructure provisioning.
◦ Familiarity with CI/CD pipelines using tools like Jenkins, GitLab, GitHub Actions, CircleCI.
◦ Experience with Docker and Kubernetes for containerizing and orchestrating applications in the cloud.
5. Programming & Scripting:
◦ Strong scripting skills in Python, Bash, or Go for automation tasks and data pipeline integrations.
6. Monitoring & Performance Tuning:
◦ Proficiency with monitoring tools like CloudWatch, Prometheus, Grafana.
◦ Expertise in optimizing data pipelines for scalability, fault tolerance, and performance.
Responsibilities:
1. Infrastructure Design & Implementation:
◦ Design and deploy scalable, fault-tolerant real-time data processing pipelines using Apache Flink and Kafka on AWS.
2. Platform Management:
◦ Manage and optimize Kafka clusters (MSK or self-managed) and Flink jobs on AWS infrastructure for low-latency, high-throughput processing.
3. Automation & CI/CD:
◦ Automate infrastructure provisioning and deployment using Terraform, CloudFormation, or other tools.
◦ Integrate new applications into CI/CD pipelines for real-time data processing.
4. Collaboration with Data Engineering Teams:
◦ Work with Data Engineers, Data Scientists, and DevOps teams to ensure smooth integration and high performance of real-time data systems.
5. Security and Compliance:
◦ Implement security for Kafka and Flink clusters, ensuring compliance with regulatory standards (e.g., GDPR, HIPAA).
6. Optimization & Troubleshooting:
◦ Optimize Kafka and Flink performance, troubleshoot issues related to message delivery, job failures, or AWS service outages.
...Busy Subaru Dealership is looking for a highly motivated employee Assistant Office Managers will assist in general office duties as assigned by the Office Manager. Assistant Office Managers will provide support to the sales and service staff, as needed. Assistant...
Class A Delivery Truck Driver JobDrives will have 3 to 4 - both home daily and overnight routes each week, delivering and unloading product... ...lift 60 lbs on a routine basis and control a hand truck with up to 400lbs on it. We use hand trucks and lift gates to deliver...
Enjoy the locum tenens lifestyle knowing Weatherby is here to support you every step of the way. Call and speak to one of our consultants today for available dates and details. ~ Must have active state license ~ BC or BE required ~ Days, Nights, 12-hour shifts...
Clinical Assistant needs 1 years experience in a customer service support role is required Clinical Assistant requires: - Possible COVID... ...interpersonal and organizational skills - Knowledge in Medical terminology - Various immunizations and/or associated medical...
...173 Thorn Hill Road, Warrendale, PA JOB SUMMARY: All drivers run daily routes with frequent stops and are required to load packages... ...in a cross-dock model picking up a loaded truck from a nightly delivery at the assigned loading dock and returning the empty truck to...