Optimize Freelance Projects with Amazon S3

Project Context:
We are building a Smart Solar Inverter IoT system. We are moving from an early-stage prototype (self-hosted on EC2) to a scalable, production-grade cloud architecture. We need a hands-on engineer who can re-architect our data pipeline for reliability, security, and cost-efficiency.

Key Responsibilities
1. Architecture Migration: Lead the transition from manual EC2-based scripts to automated, cloud-native services (AWS Lambda, IoT Core, S3, Glue).
2. IoT Data Pipeline: Design robust ingestion pipelines for high-frequency sensor data using AWS IoT Core (MQTT). Ensure data is buffered and never lost during connectivity drops.
3. Database Management: Manage and optimize time-series databases. Specifically, experience managing PostgreSQL with TimescaleDB (on EC2 or managed services) and assessing trade-offs against Amazon Timestream.
4. Security & Access: Implement strict IAM roles, VPC security groups, and private subnets. (Stop the practice of opening ports for local access).
5. DevOps & Automation: Replace manual local server "fetch" scripts with automated cloud backups (AWS Backup/Data Lifecycle Manager) and CI/CD pipelines.

Required Skills (The "Must-Haves")
1. AWS Cloud & Architecture
- Core Services: Deep expertise in EC2, S3, IAM, VPC, and Lambda.
- Infrastructure as Code (IaC): Must know Terraform or AWS CDK. Note: This is crucial to prevent engineers from manually "clicking around" and creating unrepeatable setups.
- Serverless: Experience writing event-driven code (e.g., triggering a database write when a file lands in S3).
2. IoT Domain Knowledge
- Protocols: Deep understanding of MQTT and HTTPs.
- AWS IoT: Experience with Device Shadows, IoT Rules Engine, and Greengrass.
- Edge Cases: Knowledge of how to handle device offline states, message buffering, and retries.
3. Data & Database
- Time-Series Data: Specific experience with TimescaleDB (managing extensions, retention policies, compression) OR Amazon Timestream.
- PostgreSQL: Strong SQL skills and performance tuning.
- Storage Strategies: Understanding of "Hot" vs. "Cold" storage (e.g., moving old logs from DB to S3 Glacier).
4. General Engineering
- Backend: Python (Boto3) or Node.js.
- System Admin: Linux command line expertise (for managing the EC2 DB instance properly).

• We have a Postgres/TimescaleDB database running on an EC2 instance. We need to back it up daily and make the data available for the analytics team. How would you design this workflow? Give examples from your past projects to justify

Responsibilities
• Architect and maintain AWS cloud environments, and automation programs to drive world-class agility, reliability, & security
• Design CI/CD pipelines, monitoring capabilities, IaC templates, and administrative configurations necessary to stabilize and accelerate our product delivery models
• Scale up support through automation routines by leveraging Lambda, Bash, Python, and other scripting services
• Apply veteran knowledge to maximize value created from the following services: AWS Lambda, SNS/SQS, S3, IAM, CloudWatch, CloudFormation, EC2, RDS, DynamoDB, Jenkins, AWS Service Catalog, and AWS QuickSight
• Ability to automate security policies for applications, secrets detection, and execute application security tests
• Automate and implement runtime detective controls and automated remediation scenarios.
• Configure AWS QuickSight dashboard for delivery and stability metrics
• Advise the team on best practices for peer review, daily commits, and ci/cd practices for improving security posture
• Navigate within highly agile projects

AWS Certificate Enrollment:
During the duration of the project, we need your AWS Partner enrollment under Tactical Edge. You will be requested to submit a transfer request. Upon completion of the engagement or at will, you can transfer your AWS Certifications back to your own account by following a similar process.

Required Experience
• Must required AWS Certifications Associate and Professional
• Bachelor's Degree and 4+ years of related experience in AWS Engineering for rapid prototyping / product teams
• Embodies Excellence: You aim to be a master in your craft and expect to be surrounded by colleagues who are no less than masters in their respective crafts.
• Resilient: You are not fazed by roadblocks; impossible scenarios can be overcome through brilliant ideas, solid planning, and hard work.
• Innovative: You are biased towards rejecting the status quo and actively seek novel and disruptive approaches to long-standing problems.
• Communicative: You are quick to listen and wise in speaking to promote transparent, truthful discussions.
• Technology-Focused: You are always on the lookout for opportunities to scale through software and embrace new models of getting work done.
• Design-Focused: You believe that if it’s worth doing, it’s worth doing beautifully.
• Team-Oriented: You do what is best for the team, always.

Note: freelancers only, no agencies

• Please list any active AWS certifications you have under your name? (Must be active)

We are seeking an AWS expert to assist us with critical issues in our account. Last year, a developer we hired to build a chatbot installed a set of tools on our account that have since malfunctioned and is now directing huge quantities of traffic towards our site. We haven't been able to fix the problem and as a result, these tools are effectively generating the equivalent of a DDoS attack against our site that is slowing down our site and costing us a lot of money.

We have tried to resolve the situation with AWS to no effect.

Therefore, we are looking for someone who can help us download our "good" files from AWS so we can close down our account that is malfunctioning due to the "bad" files.

jan23 Developing and testing autoscaling python AWS RAG lambda and S3 React Chatbot


Summary
PROJECT TITLE: Build AWS Hosted RAG Chatbot Using LangChain, RDS PostgreSQL with pgvector, lambda Autoscaling, and React UI hosted on S3 AWS and AWS waf and getway api for requests as users access API interface

some help
https://github.com/ed-donner/production/tree/main/week2
https://github.com/AIbyKJ/RAG_chatbot

lambda based aws rag chatbot design : frontend with S3 React, for backend lambda with python code of chatbot using langchain. aws db rds aurora sql postgress for embeddings and chunks and conversation history. lambda is stateless. python packages upload to lambda through docker file (not zip ) since python packages size maybe big (docker created on EC2 and then save to s3. lambda reads docker from s3)

do not forget that test for autoscaling proves that there is no lost requests and that as more users as more aws resources allocated or vice versa as less users as less aws resources allocated

no one resource has public access like ec2 or db (for testing users has backend access using fastapi only through httpS (not http) request with password and or current session token ). All communications inside aws.

main ideas

• Python code and AWS deployment: Backend (FastAPI) and frontend (React) are deployed on AWS and are reachable from any desktop computer and any mobile.
• Core RAG works end-to-end: Users can upload documents, content is embedded into RDS PostgreSQL + pgvector, relevant chunks are retrieved, and answers are generated via OpenAI; chat history is stored and used for retrieval with an option to forget old history.
• Multi-user + security: Admin/user authentication works; role-based access control is enforced; private vs public documents are respected; sensitive resources (DB/S3) are not publicly exposed.
• Autoscaling validated: Load test proves scaling from 0 to 120 concurrent users within 10 seconds with no lost requests (for backend and UI where applicable).
• Reproducible educational deliverables: A step-by-step .doc with screenshots for each step plus an end-to-end video enables you to recreate the full system in your AWS account (AWS Console-first so all development from AWS browser but not from local computer , no terraform or .sh).
• Local Windows tooling: Python client + load-test scripts and instructions run from local Windows to test both backend and front end.
• Full cleanup guide: Clear steps to remove all AWS resources after deployment/testing are provided.
separate material to teach me how to use react for this task
python testing code and architecture to run many queries from my windows local computer to prove it can be done as many as 120 users using app on parallel with autoscaling . from 0 to 120 users and 120 to 0 users during 10 seconds
Details are :
important
do not forget to teach me how to remove all resources after deployment or stop


important
user can access both backend (fast api ) and frontend (React ) from local computer and mobile phone

I do not know REACT so you will create separate material to teach me how to use react for this task : create architecture , code , how to change if needed. Provide both instructional video and .doc file with screen copies and explanations , and react code and how to use vibe coding for such react task. How to test such ui automatically for autoscaling testing

important all deployment should be done from AWS console (no terraform or etc ) , rds access (review , add, change records ), etc. NO use of local computer for aws setups , use local comp only for browsing aws console

defend solution from hackers, like do not provide access to DB to everyone with URL, etc

Overview
I need an experienced developer who understands Python, LangChain, and AWS to build a complete RAG chatbot system. The system must support:
1 multiple users at the same time
2 autoscaling quickly
3 store vector embeddings in Amazon RDS PostgreSQL with pgvector
3 run a clean React user interface.
4 Use OpenAI as LLM, find openai model which can support such load (so many request per minute ) by using usual subscription (not special very expensive ) .
5 Data for RAG : 5 pdf files for each user (general format pdf file for example with pictures and tables , but only textual data should be used including content from tables ), each file has 3 pages (you provide these files). both user and admin can upload pdf files , some files for public use some files only for one user , privet. You provide such synthetic pdf files as many as needed. each user upload it own files
6 users conversation history saved on RDS to use RAG of answers recent history and most suitable historical chunks from previous conversation. Relevance of chunks calculated by taking to account chunks recency as well, add capability to forget too old conversation texts
7 deploy on S3 react and lambda backend .
Make sure when there is no chatbot use (no users using chatbot ) then there is no AWS resources use (no one VM running, then no need to pay to AWS ). When user start conversation when there is no conversation at all , then resources are allocated .
8 it should be the same deployment but supports both backend and frontend request from users from local computer . it needs to test autoscaling for both frontend plus backend and only backend : one for both frontend and backend and one for only backend .

This project includes backend development, database setup, React UI, AWS infrastructure, and load testing.

SYSTEM ARCHITECTURE REQUIREMENTS

Vector Database (Managed)

Use Amazon RDS PostgreSQL

Enable pgvector extension

Store embeddings, metadata, document chunks, users, and admin data

Must be fully managed by AWS (no set up from local comp)

stateless Python Backend (backend can be used not only by aws based React, but also from local windows computer python code, you provide this code to use backend from windows and instructions how to do it , for example needed python packages installation , etc )

Use LangChain for embeddings, text splitting, retrieval, and RAG logic

Backend must be completely stateless
No local file storage
All files go to RDS or S3
Safe for autoscaling LAMBDA instances

AWS Deployment

Use LAMBDA Auto Scaling
Minimum size 1
Maximum size 120 or more

Must be able to scale up within about 5 seconds from 1 to 120 users



Use S3 for file uploads

Set up all IAM roles, security groups, VPC networking, and private subnets

React Web UI
The user interface must include the following:

Admin Panel

Admin login with username and password

Admin dashboard

View users

Reset user passwords

Upload files to the knowledge base

View basic system information

User Panel

User login

Upload personal documents

Ask questions to the chatbot

See chat history

Access control by user role

UI should be clean, simple, and responsive.

RAG CHATBOT FUNCTIONS

Core functions

Upload PDF, DOCX, or TXT files

Automatically split, embed, and store in PostgreSQL

Perform similarity search using pgvector

Retrieve top results for each query

Combine retrieved context with the LLM for answers

Support many users at the same time

Authentication

Admin login

User login

Password hashing

Store all accounts in RDS

Scalability Requirements

Must handle a quick spike in traffic

Example: from one user to twenty users very fast several seconds

Stateless application nodes

Load balanced

Health checks and automatic recovery

LOAD TESTING AND SIMULATION

The project must include tools or scripts that simulate many users using RAG running on AWS from local windows
plain python load test code running on local windows

The simulation must:

provide framework and python code for running form my local windows . Test 120 or more users logging in and use app in parallel . load changing from 0 to high during given time range for example 10 seconds. prove all works correctly , no lost requests for both backed (python ) and frontend ( React )

Upload test files

Send many chatbot messages

Trigger autoscaling

Measure response times and performance

A short report must be included with the results.

DELIVERABLES

Fully working RAG chatbot system

Hosted on AWS

Using RDS PostgreSQL with pgvector

Python backend

React frontend

Auto Scaling infrastructure

Complete codebase

Clean and well structured

Python and react code provided , including how install on local comp for local testing first

Requirements file

Deployment instructions

Full deployment guide

Important Step by step instructions to recreate the system : doc file with instructions with screen copies for each step (especially how and what aws parameters to use) and end to end video with all steps

RDS setup

IAM roles

VPC configuration

Auto Scaling Group setup

React deployment

Database schema

Environment variables

Load testing suite

Scripts for simulating many users

Instructions to run tests

Performance summary

IDEAL DEVELOPER

Strong experience with Python and LangChain

Experience with RDS PostgreSQL and pgvector

Strong AWS background
LAMBDA, Auto Scaling, VPC, IAM, S3

Experience building React apps

Experience with load testing

Ability to deliver stable production quality code

.

HOW TO APPLY
Please include:
Your relevant experience
Any improvements you recommend for the architecture

4 days to do

make sure I can run firstly python code on my local windows by using aws cli for using aws resources like rds and s3 with line by line execution for debugging ( lambda python code runs locally, then I can use copy past to use this code in lambda ) and code exploration. And after I can run fully on AWS.



Important again : you provide .doc file with step by step instructions with screen copies for each step and video how to do end to end. I should be able to reproduce all on my account by myself by using your instructions


new user can register by himself only using provided unique password in advance
for example : I give to new user passwords abc123 then new user can register without admin by using this password , then user create new unique password to login afterward

for each event like registration , login or each chat entrance log is created and saved to S3


acceptance criteria I can run on my aws account and my local windows
I do not provide aws credentials or openai credentials , you use your own credentials for development. then I will use my own accounts for testing

Create needed Backend interface with admin and users input for id and passwords
Backend interface (only very simple starting point as demo https://github.com/AIbyKJ/RAG_chatbot
)
Admin Menu
user_management_menu

("\n--- User Management ---")
("1. List users")
("2. Add user")
("3. Delete user")
("4. Reset user password")
("0. Back to main menu")


chat_management_menu

("\n--- Chat Management ---")
("1. View user chat history")
("0. Back to main menu")

data_management_menu

("\n--- Data Management ---")
("1. Upload PDF(s)")
("2. Upload all PDFs from folder")
("3. List all PDFs")
("4. Delete PDF(s)")
("5. Delete all public PDFs")
("0. Back to main menu")

vectordb_management_menu

("\n--- VectorDB Management ---")
("1. Ingest all public PDFs")
("2. Ingest PDF by filename (public or specific user)")
("3. Remove PDF data by filename")
("4. Remove PDF data by user")
("5. List available PDF data")
("6. Clear all users' memory")
("7. Clear user memory by user ID")
("0. Back to main menu")

#
("\n=== Admin Main Menu ===")
("1. User Management")
("2. Chat Management")
("3. Data Management")
("4. VectorDB Management")
("0. Exit")

USER menu
("\n=== User Main Menu ===")
("1. Upload PDFs")
("2. Upload all PDFs from folder")
("3. List my PDFs")
("4. Ingest a PDF")
("5. Ingest all my PDFs")
("6. List my ingested PDFs")
("7. Delete a PDF from storage")
("8. Delete all my PDFs from storage")
("9. Remove a PDF from vectordb")
("10. Remove all my PDFs from vectordb")
("0. Exit") less

I need assistance with setting up AWS' recommended workflow for producing an audio description track and silent control track using Lambda & Step Functions, Polly, Elemental MediaConvert, and S3 for a single 3 hour video. I also want to test with the video file with the workflow to confirm we have achieved the desired outcome. We will provide text descriptions for narration and the time stamps of where the narration should begin playing.

We need an AI / Cloud Engineer with AWS Bedrock experience to help with a small fix or configuration issue in an existing setup.

This is a quick task, not a full build.

Requirements

Experience with AWS Bedrock

Basic AWS knowledge

Able to quickly identify and fix issues

I’m looking for an experienced developer to implement an AWS-based CI/CD pipeline for an existing project. This is a small, short-term task and should take 2–4 hours.

Responsibilities:
• Set up CI/CD pipeline using AWS services
• Configure build, test, and deployment workflow
• Integrate with Git repository (GitHub / GitLab / Bitbucket)
• Ensure smooth and secure automated deployment
• Fix any minor issues during setup

Requirements:
• Strong experience with AWS CI/CD (CodePipeline, CodeBuild, CodeDeploy, or GitHub Actions with AWS)
• Experience deploying applications on AWS
• Knowledge of IAM roles and environment variables
• Clean and well-documented setup

I’m looking for an experienced developer to implement an AWS-based CI/CD pipeline for an existing project. This is a small, short-term task and should take 2–4 hours.

Responsibilities:
• Set up CI/CD pipeline using AWS services
• Configure build, test, and deployment workflow
• Integrate with Git repository (GitHub / GitLab / Bitbucket)
• Ensure smooth and secure automated deployment
• Fix any minor issues during setup

Requirements:
• Strong experience with AWS CI/CD (CodePipeline, CodeBuild, CodeDeploy, or GitHub Actions with AWS)
• Experience deploying applications on AWS
• Knowledge of IAM roles and environment variables
• Clean and well-documented setup

Lead the design, implementation, and optimization of secure, scalable, and high-performance cloud solutions for our government and private-sector clients. The ideal candidate will have deep expertise across AWS services, cloud architecture patterns, security best practices, and modern DevSecOps delivery models. This role is responsible for translating complex business and technical requirements into well-architected cloud solutions that support digital modernization, application development, data platforms, and enterprise integrations.

The AWS Solution Architect will provide leadership across cloud strategy, environment provisioning, system migration, performance optimization, disaster recovery planning, and cost governance. Responsibilities include designing end-to-end cloud architectures, implementing secure VPC/network configurations, managing IAM and least-privilege access models, enabling CI/CD pipelines, and ensuring compliance with FedRAMP, NIST, and SOC 2 requirements. The candidate will collaborate closely with software engineers, data engineers, cybersecurity teams, product owners, and client stakeholders to deliver solutions that are secure, resilient, and aligned with best practices in cloud engineering.

This position requires hands-on experience with EC2, S3, RDS/Aurora, Lambda, API Gateway, CloudFormation/Terraform, CloudWatch, CloudTrail, IAM, KMS, networking services, container orchestration (ECS/EKS), and event-driven architectures. The AWS Solution Architect will play a key role in guiding cloud modernization efforts, supporting migration of legacy systems, optimizing operational efficiency, and ensuring the long-term stability and scalability of mission-critical applications hosted in AWS.

• Describe your recent experience with similar projects
• Please list any certifications related to this project
• Describe common challenges of maintaining SLAs
• How do you create a dependency map for systems operating in multi-tier architecture?
• How do you operationalize your projects?

We are seeking an experienced AWS professional to provide hands-on guidance and support for AWS Cognito. The ideal candidate will help me understand the features, setup, and best practices for using Cognito for user authentication and management. This role involves walking me through the essential steps and answering any questions I may have during the process. A deep understanding of AWS services and user identity solutions is crucial.