PaperlayerView exact revision on arXiv

Source-linked AI summary

A Shop Floor Production Scheduling Case based on RFID-supported Smart Factory

Zhihui Chen, Yize Sun, Yuhao Dong, Zeyu Xiao, Ray Y. Zhong

arXiv:2608.16626v1cs.AI

TL;DR

Shop-floor scheduling often assumes fixed production sequences and processing times, limiting its realism under dynamic manufacturing conditions. This paper combines RFID data analysis with deep reinforcement learning and reports lower operation makespan than DQN, FIFO, LIFO, and random allocation.

  • Problem

    Existing scheduling methods commonly assume fixed production sequences and predetermined processing times, unlike dynamic shop-floor manufacturing conditions.

  • Method

    The framework mines feasible production sequences and estimates real-time production rates from RFID data before applying deep reinforcement learning for dynamic scheduling.

  • Results

    The proposed framework outperforms DQN, FIFO, LIFO, and random allocation in minimizing operation makespan.

  • Takeaways & Limitations

    RFID-informed deep reinforcement learning is feasible for dynamic shop-floor scheduling in the studied smart-manufacturing case.

Abstract

from arXiv · show

Radio frequency identification (RFID) technology has been widely implemented for real-time data collection in manufacturing shop floors, which, in turn, can be used to support dynamic shop floor production planning and scheduling. Within such an environment, uncertainty in operation and production processes collectively contribute to the dynamicity in manufacturing, thereby hampering the scheduling system from achieving maximal utility. To highlight the importance of handling such uncertainty, this paper addresses the problem of dynamic shop floor scheduling for a real-life case smart factory equipped with RFID technology. Feasible production sequence mining and real-time processing rate estimation are conducted on RFID-collected production data to quantify the operation and production uncertainties. A deep reinforcement learning approach based on the RFID data analysis is then presented for shop floor production scheduling. Simulation studies based on real-life case data have demonstrated the feasibility and practicality of the proposed dynamic production scheduling framework. Specifically, it is observed that the proposed framework outperforms existing dispatch methods in terms of minimizing operation makespan, including first in first out (FIFO), last in first out (LIFO) and deep Q network (DQN).

I. INTRODUCTION

The introduction frames shop floor scheduling as essential for reducing production-time waste and improving efficiency, while highlighting dynamic processes and limited real-time visibility as key challenges. It proposes an RFID data analysis and reinforcement-learning framework that mines feasible production sequences and estimates real-time production rates.

  • Motivation: Shop floor scheduling reduces non-processing waste, total production time cost, and delivery delays while improving operational efficiency.It assigns jobs or raw materials to individualized processing sequences to minimize time cost.
  • Problem: Scheduling complexity arises from dynamic production processes and the need for real-time shop floor visibility.Existing methods are constrained by scheduling-method limitations and limited real-time data availability.
  • Problem: Conventional scheduling commonly assumes fixed job operation sequences and predetermined machine processing times.These assumptions reflect mathematical simplification and a lack of real-time production-floor visibility.
  • RFID Background: RFID enables real-time tracking, monitoring, product identification, and shop floor production visibility through automated data collection.Smart RFID tags attached to processed materials are read automatically during production.
  • Contribution: The proposed framework combines RFID data analysis with reinforcement learning to relax conventional scheduling assumptions.It mines feasible production sequences to accommodate alternative routings and estimates real-time production rates from RFID-collected data.

II. METHODOLOGY · A. RFID data analysis for dynamic shop floor scheduling · 1) Data overview:

The methodology uses RFID-collected manufacturing data to support dynamic shop-floor scheduling analysis. The dataset captures production-process information from an automobile-engine-valve manufacturer and includes identifiers, process sequences, batch quantities, and quality counts.

  • II. METHODOLOGY: The study analyzes 413,472 records collected through RFID-enabled shop-floor production monitoring.The data came from Huaiji Dengyun Auto Parts (Holding) Co., Ltd.
  • A. RFID data analysis for dynamic shop floor scheduling: The dataset originates from a company engaged in automobile engine-valve research, production, and sales.RFID was applied on the assembly line to support production management improvement.
  • A. RFID data analysis for dynamic shop floor scheduling: RFID records reflect the whole shop-floor production process used for production management improvement.The records were collected from the company’s assembly line.
  • 1) Data overview:: Each record includes an auto-generated SQL-database ID and a batch identifier for grouping products.The fields are named ID and BatchMainID.
  • 1) Data overview:: The data identify operators and production processes through UserID and ProcCode fields.ProcCode can represent processes such as milling or drilling.
  • 1) Data overview:: ProcSeqnum records production-process order, while Quantity gives batch size values ranging from 0 to 180.The overview also defines Good Number as the number of p…
  • 1) Data overview:: The data overview includes different ProcCode sequences for production processes.These sequences are presented in Table I.

2) Feasible production sequence mining: · 3) Production rate regression analysis:

RFID data reveal interchangeable feasible production sequences for identical tasks and substantial variation in machining times. The paper therefore mines sequences and applies enriched-attribute regression, including lasso analysis, to estimate production rates for dynamic scheduling.

  • 2) Feasible production sequence mining:: Different ProcCodes under the same BatchMainID may follow different machining orders, demonstrating interchangeable feasible production sequences for the same production task.Workers may switch operation order in the real-life manufacturing case.
  • 3) Production rate regression analysis:: Machining durations are difficult to know in advance, so regression analysis using real-time RFID-collected data is used to estimate them.The approach addresses the challenge of obtaining job processing times before scheduling.
  • 3) Production rate regression analysis:: Identical ProcCodes under the same BatchMainID show significant machining-time variation, partly because rest periods and shift changes affect process-time recording.This variation motivates production-rate estimation from RFID data.
  • 3) Production rate regression analysis:: The data-processing procedure derives time-slot, holiday, weekend, and production-rate attributes from timestamp and date information.Time periods are segmented into morning, noon, afternoon, night, and midnight, while production-rate metrics are estimated through predefined equations.
  • 3) Production rate regression analysis:: The regression analysis relates attributes to response values, estimates predictor effects, and predicts responses for production-rate analysis.The response analyzed is the average production rate, with inputs including Date, Holiday, Weekend, Time Slot, and Time Cost.
  • 3) Production rate regression analysis:: Average production rate is evaluated for machine operations identified by ProcCode within designated 60-minute time slots.The analysis uses RFID-generated historical data and enriched attributes to study production-rate variation.
  • 3) Production rate regression analysis:: Lasso regression is employed to determine the significance of input variables and provide insights into production-rate estimation.The resulting sequence and processing-time insights support challenges in shop floor manufacturing scheduling.

B. DRL-based dynamic shop floor scheduling

The proposed framework uses RFID-derived production sequence and batch-rate information to support DRL-based dynamic shop floor scheduling. Deep Q learning learns an action-selection policy through interaction with the shop floor environment.

  • Framework motivation: RFID data analysis quantifies production-sequence and batch-production-rate dynamicity for the DRL-based scheduling framework.These measurements provide information for handling flexibility in the dynamic shop floor scheduling problem.
  • Scheduling formulation: Each job must visit every production-stage machine through a feasible RFID-mined sequence, while processing times are deterministic but initially unforeseeable.Production-rate sensitivity analysis estimates processing times to provide additional information for decision-making.
  • DRL-based scheduling: Deep Q learning identifies an optimal action-selection policy by learning from interaction with the shop floor environment.The DRL model performs scheduling by learning an optimal policy for the shop floor scheduling task.

1) States:

The scheduling action is selected from the observed current production state. This state is represented by a (J + 2|M|) matrix containing job-level attributes, including allocation status and remaining processing-time estimates.

  • States:: The optimal scheduling action is determined from the observation of the current state s ∈ S.The current state represents production progress in the proposed framework.
  • States:: Production progress is represented by a (J + 2|M|) matrix with 3 + M attributes for each job row.Each row corresponds to a job.
  • States:: The state includes a Boolean allocation attribute, an estimated remaining operation time from sensitivity analysis, and the true remaining operation time.The passage identifies these as attributes for each job.

2) Actions:

The scheduling agent chooses either to allocate a job to a machine or to wait, while feasibility constraints are enforced through action masking. Rewards provide immediate feedback after each state transition caused by an action.

  • Actions: Actions allocate one job Ji ∈ J to one machine mi ∈ M or wait for production.The action space contains allocation actions and a waiting action.
  • Actions: The complete action space is represented as a (|J| × |M| + 1) vector for each possible action.
  • Actions: Jobs follow feasible machine orders mined from RFID data and cannot be allocated during processing or when the target machine is occupied.A mask on Q values enforces these feasibility constraints.
  • Actions: The reward R(s, a, s′) ∈ R gives the decision maker immediate feedback after action a transitions the system from s to s′.

3) Rewards:

The reward formulation uses current makespan and indicators for allocation actions and the production end state. The deep Q network updates Q values while learning rates and epsilon-greedy exploration balance new and existing knowledge.

  • Rewards:: The reward formulation incorporates current makespan, an allocation-action indicator, and an indicator for reaching the production end state.MS_s denotes current makespan at state s; 1_A(a) and 1_S(s′) indicate allocation actions and production completion, respectively.
  • Rewards:: The deep Q network searches for an optimal scheduling policy by updating Q values during reinforcement learning.This update follows the stated DRL settings and equation (3).
  • Rewards:: The learning rate α controls how quickly new knowledge overrides old knowledge, while epsilon-greedy learning balances exploitation and exploration.With probability ϵ the agent exploits current action values; with probability 1−ϵ it explores non-greedy actions to improve value estimates.

III. NUMERICAL STUDIES · A. RFID data analysis: Feasible production sequence mining

The study analyzes RFID-collected data from a real-life smart-factory scheduling case to mine feasible production sequences for type I products. The results show that production is flexible within ordered process groups rather than requiring one fixed sequence.

  • III. NUMERICAL STUDIES: The evaluation uses a real-life scheduling case from Huaiji manufacturing company’s RFID-supported smart factory.The factory generated massive RFID-collected production data across multiple product types and shop floors.
  • III. NUMERICAL STUDIES: The analysis focuses on type I product requiring five processes: Production Codes 2, 645, 9, 705, and 213.The scheduling environment contains five machines and 20 unprocessed jobs.
  • A. RFID data analysis: Feasible production sequence mining: Feasible production sequence mining tracks each product batch in RFID-collected manufacturing data to identify viable process orders.The method targets the five required processes for type I product.
  • A. RFID data analysis: Feasible production sequence mining: The mined sequences reveal that the first three processes and last two processes can each be internally interchanged without disrupting production.This pattern is derived from observed type I product production sequences.
  • A. RFID data analysis: Feasible production sequence mining: The first three processes must be completed before the last two processes can begin.This ordering constraint is the principal structural requirement identified from the RFID data.
  • A. RFID data analysis: Feasible production sequence mining: The data-mining result indicates that type I product manufacturing does not necessarily require a fixed production sequence.Instead, feasible orders allow flexibility within the two process groups while preserving their precedence relationship.

B. RFID data analysis: Production rate regression analysis

Regression analysis examines how manufacturing factors affect machine production rates to support real-time scheduling when product finish times are unpredictable. For type I products, the analysis evaluates five production processes and achieves strong test-set fit despite a 40.23% prediction error.

  • Method and scope: Regression analysis is used to examine manufacturing factors influencing production rate and estimate machine production speed for real-time dynamic shop floor scheduling.The motivation is especially important when a product’s true finish time cannot be predicted in advance.
  • Method and scope: The sensitivity analysis covers production processes 2, 645, 9, 705, and 213 during manufacturing of type I products.Candidate predictors include work date and time slot, production location, and batch quantity.
  • Significant factors: ProcCode, temporal factors, and Quantity significantly influence machine production rate, whereas Date and Holiday show uncertain influence.Significance is assessed using T tests with P-value < 0.05.
  • Regression performance: 40.23% MAPE and 0.80 R2 are achieved by the Lasso regression on a test set containing 120 records.These metrics summarize the model’s predictive error and test-set fit.

C. DRL-based production scheduling

The proposed DQN scheduling framework uses RFID-derived production analysis to seek efficient schedules minimizing total makespan. In simulation, the network converges during training and outperforms existing dispatch methods for dynamic shop floor scheduling.

  • Framework inputs: The DQN receives estimated job processing times and current alternative production sequences to discover schedules that minimize total shop floor makespan.The scheduling experiments use a simulation environment supported by real-life shop floor data.
  • Experimental setting: The simulation case includes 15 type I jobs and 5 machines, each representing a necessary process.
  • Training convergence: Around 140 epochs are required for the DQN training process to converge, after which reward and production makespan reach stable levels.The training reward and production makespan are reported to reach an optimal or stable level around 80 epochs.
  • Scheduling performance: The proposed DRL-based dynamic shop floor scheduling framework outperforms existing dispatch methods in minimizing production makespan.

IV. CONCLUSIONS

The paper addresses uncertainty in shop floor scheduling through RFID technology and evaluates a deep reinforcement learning-based dynamic scheduling framework. The framework is demonstrated to outperform FIFO, LIFO, DQN, and random allocation dispatch methods.

  • RFID technology is used to address uncertainties in smart manufacturing shop floor scheduling.The paper explores production sequence mining and the impact of dynamic processing times on production rates.
  • The study explores the feasibility of production sequence mining and dynamic processing-time effects on production rates.
  • The proposed DRL-based dynamic shop floor scheduling framework outperforms FIFO, LIFO, DQN, and random allocation dispatch methods.The framework is demonstrated in the context of real-time data adaptation.
Loading 2608.16626v1…