Factory AI Readiness Scorecard: A 30-Minute Assessment Before Any AI or Robot Pilot

A Factory AI readiness scorecard is a 30-minute check before a factory spends money on an AI or robot pilot. It helps the team decide whether to start small, prepare first, or choose a simpler project.

In factory work, I have seen improvement projects start with the machine or software instead of the production loss. The meeting becomes a discussion about camera accuracy, robot speed, or dashboard features. The line problem is still not clear.

I would start with a different question: What repeated factory problem are we trying to reduce, and what records prove it?

This factory AI readiness scorecard does not replace engineering or safety review. It gives managers a simple way to test the process, evidence, ownership, risk, and pilot limits before the vendor meeting.

The goal is not to slow down useful technology. The goal is to protect the factory from a weak pilot that uses time, money, and operator trust without solving a real problem.

Why readiness should come before vendor selection

Factories often evaluate AI from the technology side: camera resolution, model accuracy, robot payload, edge GPU performance, cloud platform features, or dashboard design. Those details matter later. They do not matter much if the target process is unstable.

If the line changes every hour, the defect definition is unclear, operators do not follow the same method, machine data is missing, and no one owns maintenance after the pilot, even a strong technology can look weak in production.

This is especially true in labor-intensive factories. Apparel, footwear, furniture, food processing, electronics assembly, and packaging operations often have a mix of manual work, semi-automated equipment, style changes, inspection variation, and tight delivery pressure. AI does not remove that complexity. It exposes it. This is why Physical AI in smart manufacturing should be judged through operations readiness, not only technology capability.

Before choosing a vendor, use the Factory AI readiness scorecard to score the process, then decide whether the use case is ready for a pilot, needs preparation, or should be avoided for now.

How to use this scorecard in a budget meeting

Choose one possible AI or robotics use case. Do not score the whole factory at once.

Examples:

• AI visual inspection for final quality control
• defect detection after a sewing or assembly operation
• predictive maintenance for a critical machine group
• AMR movement between warehouse and production
• automated carton inspection
• operator assistant for maintenance troubleshooting
• edge AI monitoring for line downtime or bottlenecks

For each dimension, assign a score:

• 0 = not ready: unclear, unstable, missing, or high-risk
• 1 = partly ready: possible, but preparation is needed
• 2 = ready enough: suitable for a limited pilot

The maximum score is 20.

Interpretation:

• 16–20: Pilot-ready — start with a narrow scope, clear owner, and measurable success criteria.
• 10–15: Prepare first — improve process discipline, data, safety, integration, or ownership before buying.
• 0–9: Avoid for now — choose a simpler use case or stabilize the process first.

This is not a scientific certification. It is a practical filter. I would also use one stop rule: a high total score must not hide a score of 0 in safety, ownership, data access, or action control.

1. Process stability

AI works best when the process has enough repeatability for the system to learn, detect, measure, or assist.

Ask:

• Is the process performed in a similar way across shifts?
• Are work instructions clear and followed?
• Does the product flow through a predictable sequence?
• Are lighting, fixtures, tools, materials, and machine settings reasonably controlled?

Score 0 if the process changes constantly and no one can describe the standard method. Score 1 if the method exists but is inconsistent. Score 2 if the process is stable enough to test the same condition repeatedly.

What I have seen on the floor: a weak process does not become stable because AI is added. The pilot usually shows the same unclear method, missing record, and ownership gap more quickly.

2. Bottleneck or defect clarity

A good AI project starts with a specific problem. “We want AI” is not a problem. “We lose two hours per shift because operators manually check rework causes after final inspection” is closer to a problem.

Ask:

• What exact bottleneck, defect, delay, or risk are we trying to reduce?
• Can the team explain the current loss in time, cost, quality, or capacity?
• Is the issue frequent enough to justify a pilot?
• Would solving it change a real business decision?

Score 0 if the problem is vague. Score 1 if the problem is real but not quantified. Score 2 if the problem is specific, frequent, and important.

Do not use AI to search for a problem after the project has already started. Find the problem first.

3. Data availability and quality

AI needs data, but factories often underestimate what “data” means. It may be images, machine signals, PLC tags, inspection records, maintenance logs, production orders, operator inputs, barcode scans, or historical downtime reasons.

Ask:

• Does the data already exist?
• Is it stored in a usable format?
• Is it connected to the right product, lot, machine, line, time, or operator context?
• Are labels and defect names consistent?
• Is there enough sample data for normal and abnormal conditions?

Score 0 if data is missing or mostly manual memory. Score 1 if some data exists but needs cleanup. Score 2 if data is usable enough for a pilot.

For vision inspection, this also means image quality, lighting consistency, camera position, fixture repeatability, and defect labeling discipline.

4. Integration complexity

A demo can run on a laptop. A production system must connect to the factory.

Ask:

• Does the AI system need signals from machines, PLCs, scanners, cameras, MES, ERP, or SCADA?
• Is there a standard way to exchange data, such as OPC UA or another industrial protocol?
• Who will provide machine access and validate the data?
• What happens if the connection fails?

Score 0 if integration is unknown or blocked. Score 1 if integration is possible but requires engineering work. Score 2 if interfaces, owners, and fallback behavior are clear.

Integration is where many pilots become expensive. A prediction has little value if the right person does not receive it, understand it, and know what action is allowed.

5. Safety and compliance exposure

Some AI projects only advise. Others control movement, reject products, stop machines, guide operators, or interact with people. The safety question changes with the use case.

Ask:

• Could the system create a safety hazard if it is wrong?
• Does it affect machine motion, robot movement, alarms, product release, or operator behavior?
• Are there safety standards, buyer requirements, audit requirements, or compliance rules involved?
• Who approves the risk assessment?

Score 0 if safety impact is unclear. Score 1 if risk exists and needs review. Score 2 if safety boundaries and approval steps are understood.

A dashboard, inspection camera, AMR, and robot arm do not have the same risk. I would start with read-only output, then draft recommendations, then human approval. Limited automatic action should come only after the factory proves the safe boundary and fallback.

6. Maintenance and ownership

The pilot owner is not always the production owner. That gap can kill a project.

Ask:

• Who owns the system after the vendor leaves?
• Who checks alarms, retrains models, cleans cameras, updates software, replaces sensors, and handles downtime?
• Is maintenance involved before installation?
• Is IT or OT responsible for network, security, backups, and access?

Score 0 if ownership is unclear. Score 1 if ownership is discussed but not assigned. Score 2 if responsibilities are assigned before the pilot.

In real factories, I would not ask only, “Does the model work?” I would also ask, “Who will keep it working on a bad production day, and who can stop it?”

7. Changeover and variation risk

Many factory processes are not one product forever. Style, color, size, material, batch, packaging, and customer requirements change. AI systems must be tested against this variation.

Ask:

• How often does the product or style change?
• Does the defect appearance change by material, color, size, or lighting?
• Does the process change by operator skill or shift?
• Will the AI system need frequent reconfiguration?

Score 0 if variation is high and unclassified. Score 1 if variation is known but not yet tested. Score 2 if the pilot scope is narrow enough to control variation.

This is a major issue in garment factories. Fabric is flexible, defects are subtle, and style changeovers can change the entire inspection or sewing context. A controlled pilot should start with a limited product family, not every style in the factory.

8. ROI evidence

Factory AI should not be justified by excitement. It should be connected to measurable improvement; the same discipline applies when using a robot automation ROI checklist.

Ask:

• What cost, time, quality, capacity, safety, or compliance problem will improve?
• What is the current baseline?
• How will improvement be measured?
• What costs are included: hardware, software, integration, training, maintenance, downtime, and support?

Score 0 if there is no baseline. Score 1 if the benefit is plausible but incomplete. Score 2 if the baseline and measurement method are clear.

ROI does not need to be perfect before a pilot, but the team should know what evidence would make the pilot worth scaling.

9. Operator adoption

Operators and supervisors are part of the system. If the AI output is ignored, bypassed, misunderstood, or seen as a threat, technical accuracy will not be enough.

Ask:

• Who will use the AI output during the shift?
• Does it help operators do the job, or only monitor them?
• Is the interface simple enough for the real work environment?
• Are supervisors prepared to respond to alerts or recommendations?

Score 0 if users are not involved. Score 1 if users are informed but not part of design. Score 2 if operators, supervisors, maintenance, and quality teams are included early.

A useful pilot should make the next action clearer for the operator and supervisor. It should not add another screen, alarm, or target that the team does not trust.

10. Cybersecurity and data governance

Factory AI may read cameras, production records, machine signals, quality data, maintenance logs, or buyer-sensitive information. Some systems may also write to MES, release a quality decision, stop equipment, or guide a robot. Connecting the system is not the same as giving it permission to act.

Ask:

• What data can the system read, and what can it change?
• Where is the data stored, and does it leave the factory network?
• Who can access the system, change settings, or approve an action?
• Does the pilot start as read-only, draft-only, approval-required, or limited execution?
• What happens if the model, network, camera, sensor, or robot is wrong or offline?
• Are credentials, backups, updates, logs, and emergency stops controlled?

Score 0 if the data path or action permission is unknown. Score 1 if basic rules exist but the approval and fallback still need review. Score 2 if access, storage, action limits, logs, recovery, and stop responsibility are clear.

I would treat cybersecurity and action permission as part of pilot design, not paperwork after installation. Start with the smallest permission needed, keep a human approval gate, and record what the system did.

Example: scoring a visual inspection pilot

Imagine a factory wants AI inspection for a recurring visible defect. The line has stable lighting, a clear defect definition, sample images, and a quality manager who owns the result. But the MES connection is not ready, maintenance has not been assigned, and operators have not tested the workflow.

A realistic score might be:

• Process stability: 2
• Bottleneck or defect clarity: 2
• Data availability: 2
• Integration complexity: 1
• Safety/compliance: 1
• Maintenance ownership: 1
• Changeover risk: 1
• ROI evidence: 1
• Operator adoption: 1
• Cybersecurity/data governance: 1

Total: 13 out of 20.

This does not mean “do not proceed.” It means “prepare first.” The factory can still run a limited pilot, but it should not pretend the project is ready for broad deployment.

What to do after scoring

If the use case scores 16–20, define a narrow pilot:

• one line or cell
• one product family
• one defect or bottleneck
• one owner
• one success metric
• one fallback plan

If it scores 10–15, prepare before vendor selection:

• clean the data
• standardize defect definitions
• stabilize the process
• involve maintenance and IT/OT
• define the baseline
• reduce pilot scope

If it scores 0–9, choose a simpler first project. At this stage, the Factory AI readiness scorecard is not telling the team to stop improving; it is telling the team to build the conditions for a better pilot. A lower-risk automation case, such as a dashboard cleanup, inspection data discipline project, cleaning robot route test, or manual traceability improvement, may create better learning before AI or robotics.

The best first project is rarely the flashiest

I would not choose the first pilot by the best video or the newest machine. I would choose a repeated problem that the factory can measure, a team that can own the response, and a pilot small enough to stop safely.

In garment and other labor-intensive factories, style changes, flexible materials, manual handling, operator skill, and buyer requirements make this discipline even more important.

Final factory approval check

Before approving budget, ask for one problem, two weeks of records, one result owner, one success measure, one fallback, and one clear stop rule. If the team cannot answer those points, prepare the process before buying more technology.

Sources checked and claim boundary

• NIST Cybersecurity Framework: NIST Cybersecurity Framework
• NIST AI Risk Management Framework: NIST AI RMF
• NIST Manufacturing Extension Partnership: NIST Manufacturing Extension Partnership
• IFR World Robotics: IFR World Robotics
• OPC UA: OPC Foundation — OPC UA reference
• Better Work garment-factory compliance and reporting resources: Better Work reports and publications