How to Verify File Processing Status in AWS Batch | West – Convey Sandbox

Use this guide when a customer says they have uploaded files to MoveIT and wants confirmation that the files have been picked up and processed by our system. Intended for internal support agents handling AWS file processing verification tickets.

Category AWS / File Processing

Tools Used MoveIT | AWS Batch Console

Reference Ticket #26344

Overview

When a customer uploads files to MoveIT, our system automatically picks up those files and processes them through AWS Batch jobs. Your job is to confirm that:

The files were picked up from MoveIT
The AWS Batch job ran successfully
The logs show no errors

Once confirmed, you can reply to the customer letting them know everything is ready.

What Does a Customer Request Look Like?

Customers usually reach out through a web form ticket. They will typically mention the files they uploaded, the account number the files are linked to, and a scheduled test date. Here is a real example:

From: Ana Canetti / Gaspar Stanziola — Florida Power & Light (FPL)

Account #: 8513806458

Files Uploaded: lcddaily, lcdondemand, century_extract_260215100142638 (shadow database)

"Hi, we have a load control test call scheduled for tomorrow, Friday 2/27. We've uploaded all required files — lcddaily, lcdondemand, and the full shadow database — to their respective folders in MoveIT, with the load control flag set to "Y" for account #8513806458. Could you confirm they have been picked up and are ready for testing?"

Tip: When you receive a request like this, note down: (1) the account number, (2) the file names, and (3) the test date. The account number cannot be used to search directly in AWS Batch jobs, it is stored inside the uploaded file.

Important Note

If you cannot log in to AWS or see a "permission denied" error, do NOT try to fix it yourself. Contact Mike Orellana or Hardee Yu (West Team) immediately to get access set up.

Prerequisites

Before you begin, ensure you have the following:

MoveIT access - to check when the files were picked up
AWS West Prod Console access - to check batch jobs at https://prod-utilities.signin.aws.amazon.com/console
CloudWatch Logs access - to read the processing logs
The file names and account number from the customer's ticket

Step-by-Step Instructions

Follow these steps to verify that the customer's files have been picked up and processed successfully:

Check MoveIT Logs - Confirm the Files Were Picked Up: Before going to AWS, first confirm that MoveIT picked up the files. Log in to MoveIT, go to the logs section, and search for the file names mentioned by the customer (e.g., lcddaily, lcdondemand, century_extract_...). Find the timestamp showing when each file was picked up and write it down - you will use it in Step 5 to find the matching AWS job.

Tip: If a file does not appear in the MoveIT logs, it means the file was either not uploaded to the correct folder or has not been picked up yet. Check back with the customer before proceeding.
Log In to the AWS West Prod Console: Open your browser and go to https://prod-utilities.signin.aws.amazon.com/console.

If you see a Password Reset screen: Your password has expired. Enter your current password in the first field, then enter and confirm your new password, and click Confirm Password Change.

Screenshot 1 - AWS Password Reset Screen. You will see this if your password has expired. Enter old password, new password, confirm new password, then click Confirm Password Change.

Password Rules: Your new password must be at least 15 characters long, contain at least one uppercase letter (A–Z), one lowercase letter (a–z), one number (0–9), and one special character (! @ # $ % ^ & *).

Screenshot 2 - AWS Password Requirements shown in the console.

Important: If you enter the wrong old password or the new password does not meet the rules, contact Mike Orellana or Hardee Yu (West Team) to have your password reset. Do not keep trying, too many failed attempts may lock your account.
Check and Set the Correct Region: Once logged in, check the region shown in the top-right corner of the AWS Console. For West Utilities, the region must always be set to US East (N. Virginia) / US-East-1. If the wrong region is shown, click on it and select US East (N. Virginia) from the dropdown before continuing.

Screenshot 3 - AWS Console Home. Check the region in the top-right corner. Change it to US East (N. Virginia) if it shows anything else (e.g., Asia Pacific/Sydney).

Important: If you are in the wrong region, you will NOT see anything in the Batch jobs. Always verify this before proceeding.

Note: You may see an 'Access denied to servicecatalog:ListApplications' error on the home page. This can be safely ignored, it is unrelated to batch jobs and does not affect this process at all.
Go to the AWS Batch Jobs Page: Use this direct link to go straight to the Batch Jobs Advanced Search page: https://us-east-1.console.aws.amazon.com/batch/home?region=us-east-1#jobs/advanced-search

Screenshot 4 - AWS Batch Jobs Advanced Search Page. This is where you will search for the customer's file processing job.

Important: If you see a red error saying 'Could not load job queues', it means you do not have the right permissions yet. Stop here and contact Mike Orellana or Hardee Yu (West Team) to grant you access. Do not try to work around this error.

Screenshot 5 - Permission Error screen. Contact Mike Orellana or Hardee Yu (West Team) to resolve this.
Select the Job Queue and Search for Jobs: On the Batch Jobs search page, click on the Job Queue dropdown. For Shadow DB files (most common), select hvca-p-e1-batchqueue-shadow-loader. Other queues are named based on the type of job they handle. Then click the Search button.

Screenshot 6 - Select 'hvca-p-e1-batchqueue-shadow-loader' from the Job Queue dropdown, then click Search.

Tip: Need to find a specific FPL job? Type FPL in the 'Additional criteria' search box. A dropdown will appear showing job names like clients-fpl-read-file-job. Select the one that matches what you are looking for.

Screenshot 7 - Type 'FPL' in Additional Criteria to filter and find FPL-related jobs.
Find and Open the Right Job: After clicking Search, a list of jobs will appear. Look at the job names (they usually contain the file or process name) and compare the job's start time with the MoveIT pickup time you noted in Step 1 — they should be close together. Click on the job name to open its details. Inside the job details page, scroll down until you see a field called Log stream name. Click on that link to open the CloudWatch logs for that job.

Screenshot 8 - Job Details Page. Click the 'Log stream name' link (highlighted) to open the CloudWatch logs for that job.

Note: AWS Batch keeps job records for 7 days after a job finishes. If you are looking for a job older than 7 days, it will no longer be visible. In that case, ask the customer to re-upload the files or contact Hardee Yu / Mike Orellana (West Team) for older archived logs.
Read the CloudWatch Logs: After clicking the Log stream name, you will be taken to the CloudWatch Logs page. You do NOT need to read every single line. Focus on two parts:
- Start of logs (top) - Shows which file was picked up and is being processed
- End of logs (bottom) - Shows whether processing completed successfully or if there was an error
Success: If the job ran successfully, you will see this message at the very end of the logs: "Shadow process completed without errors."

Screenshot 9 — CloudWatch Logs. The last line confirms 'Shadow process completed without errors'.
Reply to the Customer: Once you have confirmed all files were processed successfully, reply to the customer with a clear confirmation. Use the Customer Response Template in the next section.

Troubleshooting & Common Issues

If you encounter issues while following the steps above, refer to this table for quick solutions.

Symptom / Error Message	Root Cause	How to Fix It
Cannot log in to AWS / Permission denied	No access or expired credentials.	Contact Mike Orellana or Hardee Yu (West Team) immediately.
"Could not load job queues" error	Insufficient permissions for AWS Batch.	Stop and contact Mike Orellana or Hardee Yu (West Team) to grant access.
File not found in MoveIT logs	File was not uploaded to the correct folder or has not been picked up yet.	Check back with the customer. Ask them to verify the correct MoveIT folder was used.
Job not visible in Batch search results	Job is older than 7 days (AWS Batch retention limit) or wrong region selected.	Verify region is US-East-1. For jobs older than 7 days, contact Hardee Yu / Mike Orellana for archived logs.
No "Shadow process completed" message in logs	Job may have failed or is still processing.	Check job status. If failed, note the error message and escalate to Hardee Yu / Mike Orellana.

Guidelines

What to Avoid

✕ Do not try to fix AWS permission errors yourself — always contact Mike Orellana or Hardee Yu (West Team).
✕ Do not search by account number in AWS Batch — the account number is stored inside the file, not as a searchable job parameter.
✕ Do not proceed to AWS without first checking MoveIT logs — the MoveIT pickup timestamp is essential for finding the correct AWS job.
✕ Do not reply to the customer before checking the end of the CloudWatch logs for the success message.
✕ Do not forget to verify the AWS region is set to US-East-1 before searching for jobs.

Customer Response Template

Once you have confirmed all files were processed successfully, use this template to reply to the customer:

Hi [Customer Name],
Thank you for your patience. We have verified that the following files have been successfully picked up from MoveIT and processed for account #[Account Number]:
• [File Name 1]
• [File Name 2]
• [File Name 3] (shadow database)
Everything is in order for the scheduled [test/event] on [Date]. Please go ahead with the test, and do not hesitate to reach out if you need any further assistance.
Thank you,
[Your Name] | Product Support Specialist

Quick Reference — Process at a Glance

Screenshot 10 - Quick Reference summary table showing all 8 steps at a glance.

#	Action	What to Do
1	Check MoveIT	Find the files in MoveIT logs and note the pickup timestamp
2	Log in to AWS	Go to prod-utilities.signin.aws.amazon.com/console — reset password if needed
3	Set Region	Verify top-right shows US East (N. Virginia) / US-East-1 before proceeding
4	Open AWS Batch Jobs	Use the direct link to the Batch Advanced Search page
5	Select Job Queue	Choose hvca-p-e1-batchqueue-shadow-loader and click Search
6	Open the Correct Job	Match the timestamp to MoveIT pickup time, click the job name, then click Log stream name
7	Read the Logs	Check start (file name) and end ("Shadow process completed without errors.")
8	Reply to Customer	Confirm all files processed successfully and test can proceed

Frequently Asked Questions (FAQs)

Q: Can I search for a job using the customer's account number?

A: No. The account number is stored inside the uploaded file, not as a searchable job parameter in AWS Batch. Searching by account number is not recommended in production as customers may upload many account numbers. Use the file name and MoveIT pickup timestamp to find the correct job instead.

Q: What if the job is older than 7 days?

A: AWS Batch keeps job records for only 7 days after a job finishes. If you are looking for a job older than 7 days, it will no longer be visible in the Batch console. In that case, ask the customer to re-upload the files, or contact Hardee Yu / Mike Orellana (West Team) for access to older archived logs.

Q: Who do I contact for AWS access or permission issues?

A: Contact Mike Orellana or Hardee Yu (West Team) for any AWS password resets, permission issues, or access problems. Do not try to work around permission errors yourself.

References & Related Tickets

For internal team use: Reference past Jira or Zendesk tickets related to this article.

ZD-26344 FPL customer (Ana Canetti / Gaspar Stanziola) requesting confirmation of MoveIT file upload and AWS Batch processing before a load control test call.
AWS Login AWS West Prod Console: https://prod-utilities.signin.aws.amazon.com/console
AWS Batch AWS Batch Jobs Advanced Search: https://us-east-1.console.aws.amazon.com/batch/home?region=us-east-1#jobs/advanced-search

Key Contacts

Name	Team	When to Contact
Mike Orellana	West Team	AWS password reset, permission issues, access problems
Hardee Yu	West Team	AWS password reset, permission issues, process guidance, archived logs