LEVERAGING AUTOMATION IN CLOUD ACCOUNT MANAGEMENT WITH AWS

In today’s fast-paced and increasingly demanding digital landscape, organizations are increasingly adopting cloud solutions to manage their IT infrastructure. Amazon Web Services (AWS) stands out as a leading cloud service provider, offering robust, scalable, and flexible solutions. However, managing multiple AWS accounts, particularly in a multi-organizational setup, can become complex and time-consuming. This is where automation practices come into play, significantly enhancing efficiency and governance.

In this article, we explore how Ritain.io, in collaboration with TEN21, tackled the challenge of AWS cloud account management through automation, resulting in a streamlined and efficient governance process.

The challenge of multi-organizational AWS cloud account management

Managing multiple AWS accounts across various organizations can be a daunting task. The need for centralized governance while maintaining segregation of workloads necessitates a robust and flexible solution. Ritain.io and TEN21 faced this challenge head-on by designing an “AWS account vending machine”.

What is the AWS account vending machine?

The AWS account vending machine is an automated system designed to simplify the creation and governance of AWS accounts. This system caters to four distinct organizational needs, ensuring efficient management and compliance across different operational areas. Here’s a detailed look at the organizations it supports:

1. Internal organization: Hosts all internal workloads, ensuring quick provisioning of resources for day-to-day business operations and internal projects.
2. Lab organization: Manages test accounts and non-production workloads, providing a sandbox environment for experimentation and testing.
3. Reseller organization: Handles client accounts for reselling purposes, ensuring rapid and accurate setup of client environments.
4. B2B customer organization: Manages accounts for business-to-business clients, offering tailored AWS resources to meet specific client needs.

The goal was to create a simple, centralized solution for account management that minimizes manual intervention and maximizes efficiency.

Architecture and implementation

The core architecture of the vending machine leverages AWS Service Catalog, AWS Lambda, DynamoDB, and Terraform, ensuring a seamless and automated workflow. Here’s how it works:

1. Service catalog and forms: Predefined forms in the AWS Service Catalog capture necessary information such as customer details, cost centers, and budget lines.
2. AWS Lambda and DynamoDB: A Lambda function processes the form data, generating an email for the new AWS account using a DynamoDB-managed email factory. This email follows a predefined naming convention, ensuring uniqueness and consistency.
3. Terraform and IAC: The infrastructure is managed using Infrastructure as Code (IAC) with Terraform. Once the email is generated, it is used in Terraform scripts to create the AWS account.

The process is entirely serverless, utilizing AWS Lambda to trigger account creation, ensuring no persistent resources are allocated unnecessarily.

Email management and security

Email management is a critical aspect of AWS account creation, as each AWS account requires a unique email address. Initially, requests for email creation were manually handled by the IT team, leading to delays and inefficiencies. The vending machine automated this process, significantly reducing the time and effort involved.

1. Email factory: The email factory in DynamoDB stores existing email addresses and generates new ones by incrementing a numerical value in the email address.
2. Security considerations: Alternate contacts for security, operations, and billing are automatically defined when creating an account, ensuring all notifications are received without relying on root user access. Additionally, service control policies are applied to deny any actions by root users in linked accounts, enhancing security.

Automation in action: creating and managing accounts

Once an email is generated and verified, the AWS account creation process is triggered automatically. This involves:

1. Terraform Integration: The email is added to Terraform scripts, which contain pre-configured modules for account creation.
2. GitHub and Atlantis: The code is versioned in GitHub and reviewed using Atlantis, facilitating collaboration and ensuring code quality.
3. Event Detection and Validation: AWS audit logs detect new account creation events. A custom Lambda function validates the account against the data in DynamoDB, ensuring consistency and accuracy.

Cost and resource management

In addition to account creation, the vending machine automates the process of generating cost-efficient reports using AWS services like Glue, Athena, and QuickSight. These reports provide comprehensive insights into AWS service usage and costs, enabling the FinOps team to identify optimization opportunities.

1. Data replication and storage: Cost data from various AWS accounts are replicated to a centralized bucket using S3 bucket replication rules and lifecycle policies to manage storage costs.
2. Data processing: Glue crawlers and Athena queries process the data, updating tables in the Data Catalog.
3. Visualization: QuickSight datasets use Athena as a data source to generate cost intelligence dashboards, providing an attractive and insightful view of AWS expenditures.

Benefits of automated cloud management in AWS

Implementing this automated solution has yielded numerous benefits:

Specific examples

The implementation of a FinOps framework can significantly enhance financial oversight and cost efficiency in cloud operations. Here are some practical use cases and examples of how FinOps can be effectively utilized within an organization:

1. FinOps Alerts: Whenever a new account is created, the FinOps team receives an alert with details about the account requester and its purpose. This immediate visibility helps manage costs effectively.
2. Service Quotas: The system monitors AWS service limits and alerts the FinOps team when thresholds are exceeded, enabling proactive cost management.

Another valuable application of FinOps is in building and managing serverless data lake architectures in AWS. This approach involves using serverless services to store, process, and analyze large datasets without managing the underlying infrastructure.

By leveraging FinOps principles, organizations can track and optimize costs associated with data storage and processing, ensuring efficient resource usage and cost control. For more information, you can refer to our detailed use case about building a serverless data lake in AWS.

Future enhancements

While the current solution is robust and efficient, there are plans to further enhance it by integrating it with the Fino product. This integration aims to facilitate the account creation process even more, allowing users to create accounts directly from the Fino interface without logging into the AWS console.

By embedding the account vending capabilities within Fino, users can benefit from a unified experience that simplifies workflow and reduces the need for AWS console access.

Additionally, this enhancement will include advanced analytics and reporting features, giving users deeper insights into their account usage and cost management directly within Fino.

Conclusion: The power of automation on AWS cloud management

The implementation of an automated AWS account vending machine by Ritain.io and TEN21 showcases the power of automation in cloud account management. By leveraging AWS services and a well-designed architecture, they have significantly improved efficiency, security, and cost management. This solution not only simplifies the account creation process but also provides real-time visibility and control, empowering organizations to manage their cloud resources more effectively.

Automation is the key to managing complex cloud environments, and this case study highlights the tangible benefits that can be achieved through thoughtful and strategic automation practices.