1 Introduction

The manufacturing horizon for Industry 4.0 [1] comprises a paradigm shift from the automated manufacturing toward an intelligent manufacturing concept. The exclusive feature in Industry 4.0 is to fulfill the real-time customer demand of variations in products in a very small lot size. This will enable a manufacturing system to meet individual customer requirement without wasting time for setup and for re-configuration of an assembly line. The intelligent manufacturing implementation may take place though the concept of internet of things (IoT) [2], in which each participating component has a specific IP address. Due to the availability of big data in IoT, the manufacturing system characteristics can be predicted precisely like predictive maintenance, robustness in product design and adaptive logistics. In this context, the smart manufacturing setup or a smart factory [3, 4] and logistics system have to fulfill the mass customization [5] demand in a flexible manner.

For a smart robotic factory to work in the context of Industry 4.0, high productivity and flexibility is the demand of the future. To cope with this issue, robots may take most of the workshare in future manufacturing, yet the human worker has to stay in the work area either in supervision role or for the jobs for which the robots cannot be trained. The constant human presence in or near the work area of intelligent robot leads to a shift regarding safety. The conventional approach is to expose human workers up to a limited extent to the robot and with appropriate safety control that leads to full stoppage (safe hold) of a machine in case of worker violation of the robot workspace. This causes interruptions and resetting procedures to be activated which reduces productivity. The futuristic approach is to implement robotic applications where robot and human workers can coexist and collaborate safely. In this setting, the robots share the same workspace with human counterparts and perform activities like raw material handling, assembly and industrial goods transfer.

Due to the presence of more than one million conventional (non-collaborative) working robots in the industry [6], converting the present day conventional robots to collaborative ones presents a lot of revenue potential. These conventional robots cannot be replaced with new collaborative robots (see Table 1) in manufacturing areas because of the huge financial cost involved. One approach to convert these conventional robots into collaborative ones is by making their environment intelligent, e.g., by putting sensors around the robot working area in addition to the capturing of human worker motion. This way, multiple conventional robots will be able to collaborate with humans. This will be an advantage for the manufacturers as capital investment on newly developed collaborative robots may not be required. To establish such a collaborative environment, a cyber physical system (CPS) needs to be established which takes care of all the necessary requirements of communication, safety, security, sensors and electronics. This will also allow even very large payload robots to carry out the tasks in a collaborative manner as is the case in the small to medium payload robots shown in Table 1. In one such attempt in MIT [7], the human motion capturing sensors are used with a non-collaborative robot. The virtual component resembling the actual scenario of man and robot is used to calculate the distance between the robot and the human. Based on the real-time distance calculation, the robot controller is given the task by an external module to systematically reduce the speed. This way, a generalized solution is sought to make a conventional robot intelligent.

2 State of the art in collaborative robotics

The state of the art development in collaborative robotics has roots in the technologies arriving from the humanoid robotics, artificial intelligence and exoskeletons, which were developed over the last two decades. The basic objective of such robotic humanoids is to work in household and medical applications to attend the needs of disabled and old people. In the industrial domain, there is only a recent trend for the development of intelligent collaborative robots. Table 1 shows many examples for such collaborative robots that can work alongside humans without creating hazardous situations. So far, the collaborative robotics is developing fast in industry and it is estimated that the collaborative robotics sector will grow to US$1 billion by 2020 [6, 8]. This growth is driven by small to medium manufacturing, electronic manufacturing and allied services provider companies. For industries looking for such agile manufacturing technologies, robot manufacturers develop collaborative robot designs which are suited for small- to medium-size product handling and other operations.

Table 1 State of the art collaborative robots
Full size table

In Table 1, multiple examples show dexterous robots comprising of single or dual arms that have multiple degrees of freedom (DOF). In most of the cases, the tool end-effector repeatability shows the capabilities of modern collaborative robots to handle intricate tasks. All of these robots can work and collide gently with humans on the factory floor as the joints are developed with internal force sensors. The arms and heads are equipped with high-resolution cameras, even 3D cameras for tracking. In some cases [15–17], visual markers are used for fast recognition and tracking, on every tool which are needed to the robot to complete the job. All the robots have programmable compliance, such that they can be trained for the new job on the shop floor. Yet, the maximum payload capacity varies from 0.5 to 14 kg, i.e., small- to medium-sized payload. Collision detection, instant hold upon collision and speed reduction upon violation of workspace are the common implemented technology features. It seems that there is a paradigm shift in the role of robots in industry and services from conventional unintelligent robots to collaborative ones. Also, these recent developments range from small- to medium-scale payload applications in human–robot collaboration (HRC), paving the way for heavy robots to become collaborative as a next step in industrial collaborative robotics. A very recent example is of FANUC’s CR-35iA [18] capable of carrying 35 kg payload with category 3, performance level (PL) (d) safety certification, according to ISO 10218-1:2011.

The paper has two basic objectives: The first aim is to identify the collaborative schemes and formulate a formal grading system; secondly, to define a CPS for human–robot collaboration in industrial scenarios and develop a methodology that can search for appropriate solutions in a given industrial scenario down to sensor level. The latter allows us to convert conventional heavy payload robots to intelligent ones for any industrial setup. Further detailed considerations for an equipped external environment for such robots are derived from pre-defined safe CPS according to the scenario requirement and collaboration level sought. The approach is initiated by studying implemented robot safety schemes and then evolving effective collaboration schemes. Once the collaborative schemes are sorted, some key indicators are introduced for formal categorization of industrial collaborative scenarios with examples of few selected sensors. A hypothetical collaborative example is presented to identify the sensor level requirements for a given industrial scenario. The paper is summarized with a design methodology for the development of such CPS in the context of variation in industrial scenarios.

3 CPS in human robot collaboration

The proposed approach is to exhibit safe intermediate HRC without passive safety mechanisms (e.g., fencing). In order to realize this, extra safety and protection measures need to be implemented for a collaborative robotic CPS (CRCPS). These safety and security (protection) requirements are based on the level of interaction between humans and robots on the shop floor to increase productivity. Security is moreover closely related to safety as both these system level properties have to be considered concurrently. Security essentially protects the systems from humans as attackers and the safety physically protects humans from the systems (e.g., avoiding collisions). In fact, the approach in the design of CRCPS is to merge the safety and security concerns just like designing industrial facility, control and risk assessment that consider both aspects [19]. However, in this paper, only the safety aspects are considered for CRCPS development because security can be studied in this specific case only once a safe HRC system is ensured. Security is left as the future direction of current research on CRCPS development to secure a ‘safe HRC system’ from the cyber-attacks.

A CPS is a smart system in which the computational and physical systems are integrated to control and sense the changing state of real-world variables [20]. The success of such CPS relies on the sensor network and communication technologies that are reliable, safe and secure. In CPS, all the functional components are in modules and interconnected (wirelessly) in the production line or in the smart factory. Even raw materials and machines are connected to the network cooperating with human workers through human–machine interaction (HMI) systems. Hence, the CPS platform evolves its architecture to engineer across the digital–physical divide and removing the borders among the key technologies. In particular, the CPS for manufacturing and production [21–29] may consist of electronics, computing, communications, sensing, actuation or robot, embedded systems and sensor networks. The CPS in manufacturing needs other resources like flexibility of the manufacturing system, the manufacturing scenario and the adaptability of changing assembly tasks [30], in addition to HMI technologies and other typical CPS modules. For the application in HRC, the deployment of a full scale CPS accounts for the human worker as an inherent part of the system. To state the CRCPS definition, the three components are clearly evident in the model with detailed adaptor modules (see Fig. 1). The CRCPS structure is inspired by anthropocentric CPS (ACPS) [16, 29, 31], mainly due to the cohesion of the human as an inherent module.

Fig. 1
figure 1

Structure of CRCPS: detailed components, modules, adaptor technology modules and interconnected links

Full size image

The human component (HC), the physical component (PC) and the computational component (CC) represent the three main integrated entities. The interaction among the three entities depends upon the advent of the enabling adaptor technologies. The HC is well connected through different adaptor technologies, e.g., accurate human position tracking technology is essential adaptor in the CRCPS. The CRCPS is a highly automated system as it removes the boundaries between the composite elements, thus preferring their operational interactions. There are various HMI technologies based on human senses of vision, acoustics and haptics. The proposed CRCPS can utilize vision system for detection, tracking and gesture recognition of human workers. The robots can also be commanded using acoustic signals from humans (e.g., voice control). Additionally, a variety of sensors and actuators can provide the interaction between HC, CC and PC. There are standard interactions shown between the components which have to contribute with a role. Adaptor technologies are scenario dependent and can be seen as plug and play devices. There are other optional scenario-dependent interactions between the standard components and adaptors in CRCPS.

The CRCPS is an extension of the CPS and for that reason must show compliance to the system level properties of a CPS. For this, CRCPS must exhibit properties like integrality, sociability, locality and irreversibility. Moreover, it must be adaptive, autonomous and highly automated [32]. Integrality for CRCPS means that its functional components are well integrated to perform self-organizing tasks like learning and adaptation. The ability of CPS to interact with other CPS through different communication technologies defines the sociability. It will encompass not only devices but also integrates humans as well. As an example, if the two CRCPS are functioning in a close physical distance, then the worker belonging to a CRCPS must be able to interact safely with the robot that belongs to the other CRCPS. Locality introduces the computational, human and physical capabilities of a CPS, as bounded by spatial properties of the environment. Irreversibility of the CPS makes it self-referential in timescale and state-space. The adaptive characteristic makes the system self-organized and evolving. The autonomy [16] refers to the roles of functional components and the CPS itself as capable to make independent decisions.

4 Collaboration classification

For CRCPS industrial environment, a smooth overlapping of workspace zones of robots and humans is considered in which both can interact. The formal grading of the human–robot collaboration involves the level of interaction between the two entities. The level of interaction can be formalized based on the distance between the two entities, workspace share level and the complexity of collaborative tasks which both are performing mutually. Many human avoidance schemes based on human activity prediction or human and robot position estimation at the same time [33–35], risk prediction control [36] and augmented reality [37] are considered to be implementable in an interactive environment. There are also fatalities reported [38] in countries where usage of robots is intensive despite putting all the safety and protection protocols. For example, in Germany, such accidents range from 3 to 15 annually from 2005 to 2012. Note that this rate relates to accidents without any collaboration between humans and robots. There is also an issue of mental strain on humans in addition to the physical interaction of robot and human. It is discussed by Arai et al. [39] that by restricting the moving area and moving speed of robots, the mental strain of a human operator remains low. Also, the prior accurate information of robot motion is essential to decrease the strain on a human operator. In this context, there is general need to classify the collaboration level and specific to heavy payloads, it is obligatory to reduce the level of risk in HRC.

To formally grade the HRC, the safety approaches in practice must be known first. All the examples shown in Table 1 follow at least one safety approach during human robot interaction. Safety schemes based on position prediction and building intelligent environment [40] around robots are summarized. The intelligent environment means to equip the robot environment with appropriate monitoring sensors to make it aware of situation, human, safety zone and distance. However, the four basic principles of safety protection of working with robots are described in [41, 42]. Here, these approaches are outlined briefly.

A common approach using small size robots is to provide guidance manually or reduce the robot speed as per requirement. This manual approach is open loop, without sensing, has high HRC level, is restricted to small size robots and depends on the defined risk assessment. The basic safety approach can be termed as ‘complete isolation’. In this approach, a specified work zone is covered with sensors like laser scanner or proximity sensor. In this case, the robots must stop at the human access to the work area. These systems are sensor dependent, closed loop and have almost no HRC level attainment (see Fig. 2 for collaboration schemes).

Fig. 2
figure 2

Collaboration classifications: a robot on safe hold against human violation, b speed reduction if the worker is in the robot work zone, c robot touching the human with a pre-defined calibrated force

Full size image

The third approach is the speed and separation monitoring through vision-based systems or other possible techniques. Speed reduction schemes of robot can be applied with a possible stop or speed reduction in case of worker enters the dangerous zone. This safety concept uses multiple integrated sensors and an effective sensor fusion technique to develop a fast, reliable real-time monitoring solution for HRC. High HRC level attainment is possible but poses challenges to the risk assessment in case of a failure of a monitoring function. The speed monitoring can be integrated with separation monitoring, in which human avoidance algorithms are used in a dynamic human tracking context. A small active area around the human position is marked and continuously updated for the human motion in the robot work zone, forcing the robot to actively avoid such a space. The last concept is the force monitoring through the use of force sensors. This system will also work with the help of a vision field which will guide the robot in case of a human presence. The robot speed and acceleration reduction will take place according to the level of force allowed to hit a specific part of the worker’s body. This scheme demands integration of force sensors in addition to the sensor technology required for basic area monitoring. The scheme provides highest level of HRC attainment but poses a challenge to the risk assessment in case of failure of any monitoring function.

By looking at different collaboration techniques, it is possible to categorize these by several parameters. Figure 3 shows the collaboration level from low to high. There are four equally weighted key performance indicators (KPIs) selected to contribute in the overall HRC grading scheme. These indices are PL, safety distance (SD), risk (R) and the reaction time (RT). PL is taken as the ‘mean time to dangerous failure’ (MTTFd) and defined in the EN ISO 13849-1 based on the average number of cycles per year until 10% of the components have a dangerous failure.

$${\text{MTTF}}_{\text{d}} = \frac{1}{10}E(x)\quad {\text{for}}\,x = \hbox{min} \{ P({\text{failure}}\,{\text{of part}} \cdot {\text{cycle/year}}) = 1\}$$
(1)

E is the mean time until 10% of the components have a dangerous failure or the component operating time is restricted to E. The units of this indicator are expressed in years, e.g., the MTTFd range for electromechanical components is 100–200 years. This means that the component needs replacement after 10% of the MTTFd value. A period of 20 years as a component replacement time is set as a goal according to the standard and can be taken as the maximum value for this indicator.

Fig. 3
figure 3

HRC grading scheme: four KPIs on the left and grading calculation is on the right side

Full size image

The second indicator is the SD calculated between human and a working robot. The SD formula for a human working with an industrial robot is given in EN ISO 13855.

$${\text{SD}} = (K \cdot T) + C$$
(2)

SD computes the minimum SD from the risk zone. K is the speed of the man approaching to collision with the robot (mm/s). T is the robot’s follow-up time in (s) to stop completely, once the brakes are applied. C is the additional distance (mm) for safety compliance that depends on the sensor’s capability or resolution. In case of multiple sensors used in a system, the sensor with lowest resolution can decide the resolution of the overall system if any sensor fusion technique is not used. Calculations with various sensors show that SD = 0.5 m is not possible even with very fast sensors (see Table 3). Yet, no human robot collaboration can be implemented if SD is larger than approximately 2 m.

The 3rd indicator is calculated based on the manufacturer specifications according to the number of unsafe components used and is termed as risk (R).

$$R = \frac{U}{U + S}$$
(3)

The ratio of the number of unsafe components (U) divided by the total number of components is referred as risk (see Eq. 3), where S is the number of safe components used in the system. According to the PL range specified above, i.e., electromechanical component below 100 years MTTFd, is considered unsafe. Additionally, each sensor product itself may be specified based on the number count of safe and unsafe components used. In a CRCPS perspective, all the used sensor components and equipment can be marked safe or unsafe. The minimum risk can be specified when all the components used are safe. The maximum risk can be checked according to a benchmark or left on the designer’s disposal or risk assessment based on the ISO 12100:2010.

The fourth index to gauge the effectiveness in collaboration is through the data delay rate (D i ) of the sensors (see Eq. 4). The diversity of sensors used in a designed CRCPS may have an asynchronous data transmission rates. Data delay rates (ms) are important as the delay time from every sensor counts on the overall system’s RT to respond in a case where any sensor fusion technique is not used. Thus, the overall delay time of the system is the key indicator, enabling the robot to initiate the safety protocol in time to avoid any hazard. Larger delay time can affect the robot’s RT adversely and hence reduce the effective HRC attainment. The other variable is the number of sensors (N) installed in a CPS system. In the case, the system consists of a number of heterogeneous sensors, this variable represents the number count of slowest sensors. Here, k is a constant. It is zero for a completely isolated systems and one for all other monitored systems.

$${\text{RT}} = k(D_{i} \cdot N)$$
(4)

After looking at different collaboration techniques and the performance indicators, we now formalize the collaboration grading scheme. Figure 3 shows the grading pattern of HRC from low to high. On the right side in Fig. 3, the HRC grades are specified, where a1 shows the highest level of HRC attainment and d 2 the lowest.

All of the above-mentioned indicators are converted to the corresponding indices on the scale of 0–1 (divide by the best KPI value).

$$I_{j} = \frac{{{\text{KPI}}_{j} }}{{\left( {{\text{KPI}}_{j} } \right)_{b} }}$$
(5)

In case of SDI and RTI, inverse scale is used as the best values are the smallest, e.g., 0.5 m for SD calculation is very difficult to achieve. On the right side in Fig. 3, the collaboration grading is specified based on the sum of all the four indices with equal weights, resulting in a maximum score of 4. Here, ‘a’, ‘b’, ‘c’ and ‘d’ correspond to a scale of 3–4, 2–3, 1–2 and 0–1, respectively. This way, the collaboration attainment is divided into four large categories, where each category is comprised of two sub-categories.

5 Sensors catalog

To assess the HRC attainment level, it is necessary to compute all the KPIs for a given collaboration context. For this purpose, the safety schemes and the possible risk-reduction approach mentioned above are further explained at the sensor level. The HRC schemes are studied to incorporate sensor level requirements of the CRCPS and generate a sensor catalog for each type of collaboration. The sensor catalog is a sensor library that can be established with various sensors of diverse specifications and can be integrated in the design methodology of the CRCPS. This catalog together with performance indicators forms the basis of an optimization algorithm to generate a list of possible feasible solutions for any given industrial scenario. It may also reveal nonexistence of any feasible solution. One of the basic conditions for CRCPS implementation is the known positions of human and robot in real time. In some cases, it is also important to know the extents of the assembly and the scenario for the operation.

Scenarios are the possible situations in which an industrial process can take place, e.g., a large automobile engine held by the robot gripper is presented to the worker for an industrial process like quality inspection, drilling, seal adhesion, fastening. [43]. In any given scenario, real-time location information of body parts of the worker is important. For example, a motion sensor installed on an arm can give real-time information about the arm position. Yet, if the position information of worker hand is not included, the estimation of assembly size and worker hand size must be taken into account. A different example is of vision sensors employed for the position information of human worker which must be workable in different lighting conditions, e.g., in low visibility or in a rough industrial environment. Similarly, the communication must be fast enough for an immediate and accurate response of the robot which exemplarily could be the case for a low distance, safe wireless network. Overall, the system must comply the relevant safety standards like EN ISO 13849-Part 1 and 2 and EN ISO 13855. These standards provide principles, safety requirements and guidance for the design and integration of safety-related parts of control systems.

Table 2 defines the collaboration level of different safety approaches, employable risk-reduction schemes and the basic sensor pack currently available, to implement a safety concept. The solutions can be found based on the industrial scenario and HRC level sought. For the speed and separation monitoring case, inertial measurement units (IMU) are employed in addition to the basic area and position monitoring sensor systems. Active human avoidance algorithms are part of the solutions in addition to the applied sensors. Similarly for force-monitoring-based HRC system, the basic area and position monitoring will be a requirement for implementation of the CRCPS in addition to the force sensors. In force monitoring, different types of geometry adapted tactile sensors are available to be installed at the robot joints, with shock-absorbing properties for safe collision detection and touch-based interaction. Force sensors of different force ranges can be used for assessment of force exposure limits for different human body organs. However, use of force sensors in robot joints is a new trend in collaborative robotics as shown in Table 1.

Table 2 Collaboration concepts and required technologies
Full size table

Table 3 shows the computation of some indices that are checked for different employable sensors in the CRCPS. While these data were obtained for specific sensors only, it may still be regarded to hold similarity for sensors of these classes. The number of sensors is selected according to the practical requirement for such a system. For example, to check the worker entry into the robot workspace, only one laser scanner is required. In order to monitor the worker position through a vision system, a minimum of two cameras is needed for full field coverage. Moreover, to design a worker vest, a total of four IMU`s are required at minimum to cover the body front, back and arms.

Table 3 Indices computation for sensors: security laser scanner, time of flight camera, motion tracking inertial measurement unit and quality assist ultrasonic sensor
Full size table

It is noted that the SD is large in case of a camera system as compared to other sensors that makes HRC nearly impossible. Moreover, SD = 1 m is required in any case for the deployment of safety speed reduction scheme, e.g., if a worker is coming toward a robot with a speed of 1600 mm/s and the robot’s follow-up time is 0.42 s, then the robot must exhibit safety speed reduction when SD <1 m. For the RT calculation, ultrasonic sensors show the best result.

6 Hypothetical application scenario

The core of the CRCPS development is the integration of dynamic characteristics of the individual components. The individual protection components register context, situation, and status of worker, machine, plant, and process and activate protective mechanisms before a hazard, e.g., a collision, can occur. The production process will run without threats and interruptions and will achieve the level of security and safety meeting legal requirements on an industrial floor. Symbiotic human–robot collaboration [32] is defined for a fenceless environment, in which productivity and resource effectiveness can be improved by combining the flexibility of humans and the accuracy of machines. CRCPS can enable such HRC with the characteristics of dynamic task planning, active collision avoidance, computational intelligence [44] and adaptive robot control. Humans are part of the CRCPS design in which human instructions to robots by speech, signs, hand gestures or other adaptor technology are possible during collaborative handling, assembly, packaging, processing or other tasks. All of these industrial tasks require a solution for HRC specifically in the domain of conventional medium and heavy payload robots, as there is no such solution exists so far.

Figure 4 shows a monitored area in which a human and a robot are interacting for completion of an industrial task. The vision system can be established through overhead 2D cameras or a 3D stereo vision camera and an additional laser scanner to cover any violation of robot workspace by a human worker. The vision system is providing the real-time location information of the worker, to the system. The robot system is programmed to reveal its end-effector position in all six DOF. The vest, which the worker will wear all the time, contains multiple IMU fitted at various body locations of the human worker thereby providing position and rate information to the CRCPS. The same can be proposed for an IMU fitted helmet for accurate head positioning information. These IMUs contain six sensors, i.e., three gyros for the three angular deflections and three accelerometers for linear acceleration measurement.

Fig. 4
figure 4

HRC in CRCPS design: a hypothetical industrial scenario example

Full size image

A pre-defined safe distance margin enables the system to identify if the worker is near to the robot. Speed or acceleration reduction can be started suddenly upon identification of a dangerous situation and may lead to full stoppage of the robot until worker leaves the safe distance limit in the workspace. The robot will continue its job from the point it went in to full stop. There is an interaction mode in which either the hands or the worker voice can be utilized to train the robot. For this, different hand gestures can be used to train the robot in the interactive environment. For force-monitoring system, force reduction approach is applied suddenly, once the SD margin is reached. Force sensors can provide an additional feature in the case of touching the human worker. Force calibration for different body organs is a must requirement in order to design such systems. Joints of new collaborative robots are equipped with force sensors, torque sensors and load cells. However, conventional robots without force sensors in joints cannot be used in the force reduction and monitoring approach. Collaborative robots as shown in Table 1 have the capability of collision detection and hold operation once collided with human worker. Force calibration on the basis of collision forces that are below any threshold of human pain level is required. There are recently developed [45–47] guidelines on contact forces based on biomechanical experimentation.

In such a CRCPS, multiple sensors integration and computational intelligence schemes like human tracking, human avoidance and intelligent use of multiple sensory data can be implemented. Due to the resource exhaustive nature, the real time and software issues arise in the embedded systems distributed intelligence. The integration between the cyber and physical layer requires communication and synchronization of the embedded system software that introduces complexity, limiting performance of real-time system [48] and the emerging problems due to the compromised cyber-security during the product life cycle. To cater for such issues, there are overhead controlling and self-verification approaches [49–51]. Such approaches can be useful in dealing with unusual system behavior within CPS modules and to find out the actual cause of the malfunction. These system integration approaches in CPS research include intelligent sensor fusion techniques, intelligent modular synchronization and different layers of protection checks and verification schemes depending upon the allowed overhead.

7 Generalized methodology for various industrial scenarios

In addition to the safety concepts, HRC attainment level and the sensor technology employed for a particular solution, there may be multiple industrial scenarios for which a generalized methodology can be established. Figure 5 shows the general methodology for building a CRCPS in a given industrial scenario. The methodology starts from an HRC industrial scenario from which the detailed customer requirements are generated. The methodology shows criteria based on several collaboration indices. The indices are evaluated based on the sensor level information from the sensor library that can be established on the basis of state of the art sensor technology and holds vital specifications information in a software form. Once an initial set of sensors is selected, an optimized solution is searched between the collaboration indices and the sensor specifications selected from the library. The final solution of the optimization algorithm is matched to customer-specific requirements for the CRCPS design. If the result is unfeasible, the requirements are then adjusted according to the presented solution. Once customer requirements are met, the solution is implemented.

Fig. 5
figure 5

Methodology for implementation of CRCPS in various industrial scenarios: key components are the sensor’s library and collaboration indices

Full size image

Figure 6 shows the optimization procedure in the design methodology for CRCPS. Detailed sensor specifications from the sensor library are used to tabulate the initial data from the selected sensors. The upper and lower bounds of the specifications are set as part of the data input. These are the input constraints applied and can be changed by the user if the final optimized solution does not come up to the customer requirements and expectations. The initial data are populated using a suitable design of experiments (DOE) technique, e.g., factorial method, Taguchi or randomization. After spreading the initial population, multi-objective genetic algorithm runs that is selected due to the characteristics of directional crossover, fast convergence and objective function penalization. Multiple objective functions are defined according to the collaboration indices or KPIs mentioned in Fig. 3. In the optimization process, total number of iterations t is calculated according to the size of the initial population times the selected number of generations (N gen). Once the number of iterations reaches N gen, the algorithm stops and presents the final solution for technology selection. The optimized solution is the set of sensor specifications that can achieve best possible KPIs. By using optimization algorithms like genetic algorithm or optimization techniques using heuristics, the global optimum can be reached in the final solution that can avoid the local optimum traps.

Fig. 6
figure 6

Optimization data flow in CRCPS design methodology

Full size image

Apart from the data flow in the CRCPS methodology, there can be various industrial scenarios based on real-life industrial situations like in an assembly line in which a single worker may interact with multiple robots or vice versa. There can be technology solutions, other than camera systems for the scenarios of varying illumination conditions at different day timings. The systematic evolution of scenarios is based on the technologies delivering worker position information in the CRCPS as the robot gripper position is known from any scenario. Table 4 summarizes those technologies comprising of sensor systems and software algorithms that are required to complete basic industrial tasks. An intelligent multiple worker tracking system is an example of software modules in addition to the camera systems in case of multiple workers interacting with robot at the same time.

Table 4 Human position recognition systems in different industrial scenarios
Full size table

Figure 7 shows an industrial scenario where the activities are carried out in varying illumination conditions in different parts of the day. This lighting condition normally exists in small and medium enterprises where the factory floor is not completely isolated from the outside environment. In this case, day light camera systems can be compromised to identify the worker position; however, other technologies like radar system and IMU can function normally. Figure 8 shows a scenario in which multiple workers are collaborating with the robot at the same time. For this to implement, a smart multiple tracker needs to be developed. The intelligent tracking system can work for both, radar and camera system technologies, i.e., in this scenario, every worker collaborating with robot must wear a sensor equipped vest, band or helmet all the time. The intelligent tracking system must be able to deal the multiple workers separately. The technical scenario may get difficult in the case of physical obstacles and the two workers coming very close to each other hiding the tracking markers of one of the two. These special cases must be dealt with, by advanced estimation schemes and filters implemented in the tracking system.

Fig. 7
figure 7

Industrial scenario with varying illumination conditions

Full size image
Fig. 8
figure 8

Industrial scenario with multiple workers needs multiple worker identification tracker

Full size image

The above-mentioned industrial scenarios are summarized in Table 4. Although a generalized solution cannot be presented here, the table with some variations can be considered as generic for industrial scenarios which range from single robot to multiple robots working together with multiple human workers. The possible approaches in order to build HRC system may include options like the use of inertial sensors, vision, radar or any hybrid approach for the human position monitoring. The hybrid option may consider any of the two approaches in a combined way. The real benefit of the hybrid approach is the execution of tasks with high precision as the positioning information from two separate sensor systems will work mutually and compensate for the errors. Additionally, one technology area might be more practical in a given scenario like the vision system can be compromised in some conditions or in the case of vision obstacle. In that scenario, the other sensor technology will keep the system functional.

8 Conclusion

The paper identifies the requirements of HRC in an industrial context. A controlling CRCPS structure for HRC suggests the human worker to be an integrated part for which various interactive technologies can be employed. This approach is different from the common CPS concept in which there are only two components and the cyber component controls the physical system. There is also a definition of ACPS in which the HC takes the integrated role with the cyber and PCs. Also, in the ACPS structure, no role is fixed for any component and the roles take the shape as the CPS learns from the tasks and activities. The ACPS concept is best suited to the machine learning environment, humanoid robotics, small-sized collaborative robotics and other artificial intelligence applications, where the human role is important to train the robot for new jobs or for programmable compliance. For CRCPS, the concept of considering human worker as an integral part of the system and teaching the heavy payload robot for new tasks can be a similar concept. However, the flexibility part in the ACPS model makes it different from the proposed CRCPS structure where the human role is integrated, but with less flexibility. One example of a less flexible human role can be seen where the intelligent heavy payload robots may be restricted from programmable compliance due to safety reasons and the workers dealing with such robots have to use programming pad to train robots for new jobs. This is the basic difference in the concepts that differentiate CRCPS from ACPS.

It is found that state of the art collaborative robots is being developed for small- and medium-sized manufacturing applications and the solutions to convert for large number of existing conventional robots to intelligent ones are in infancy stage. Also, the unintelligent heavy payload robots do not have a collaborative solution yet. In this context, different safety approaches of working with robots are studied and potential detailed solutions are identified. Although, it is difficult to identify and integrate all the collaboration and industrial process requirements, but some performance indices are established for collaboration like PL, SD, risk of using unsafe components and RT based on data delay rate. These indices are realized with their best and worst values to form a scale from 0 to 1. A sum of these KPIs based on equal weights is used to reveal a grading pattern in HRC. However, it is important to further study unequal weights according to the role of every KPI in the collaborative process. An example is SD because this parameter has a decisive role in defining collaboration. HRC cannot take place at all without having the SD value within the prescribed range. Even, it can be stated that other performance indicators are valid only, if the condition for SD is met. There can be other important KPIs which may contribute in the grading system. Those KPIs can be evolved from the requirements of the identified collaborative process. Process KPIs depend upon the complexity, nature and duration of the industrial collaborative task. A general mapping of process KPIs and integration with the existing grading system is the future extension of this work.

Safety schemes are graded for HRC attainment level, and the sensor level requirements for each collaborative mode are identified. It is found that the force-limiting approach has the highest HRC level but it is only possible with force sensors in joints of the robots. Joints with force sensors are not the common feature in conventional robots, thus force-limiting approach cannot be applied to convert more than million conventional working robots in industry to intelligent ones. Speed and separation monitoring are the appropriate approaches that can be utilized in the case of conventional robots.

The paper is also focused on the technology limits and real-time issues for the achievement of high HRC. A collaborative scenario example revealed the detailed sensor requirements of a hypothetical industrial HRC system in which IMU, camera system and laser scanner are the minimum required technological components. For CRCPS design in HRC, a generalized methodology is developed to cater for various industrial scenarios. The methodology shows that a sensor library has to be established and an optimized solution can be searched using collaboration indices and sensor’s specifications. The library is suggested as the sensor specifications are stored in a standard soft form. The library can be further classified into different categories of sensors so that two sensors of the same type but with different specifications can be placed in a unique category. Although the optimized solution will show the best possible available sensors which suits the given industrial scenario, but the user can apply and modify the input bounds on the sensor specifications, if required in the proposed methodology. However, it is challenging to maintain an up to date sensor catalog from various sources and manufacturers. The methodology also mentions the use of unsafe components as a key index. Using a large pool of database for the component selection, the number of unsafe components in a solution can be increased from a certain critical level. The usage of such components will also hamper the industrial certification process of such HRC system.