KINE Robotics Reduces Costs with Accurate Conveyor Tracking System
KINE Robotics Oy provides custom turnkey robotics solutions to customers in a variety of industries, including food and beverage, logistics and general manufacturing. They design and implement customised robot-based solutions to meet their clients' needs.
Zebra Success Story: KINE Robotics
Overview: Manufacturing Challenge
Develop a conveyor tracking system to accurately pick transparent bags of bread and rolls that come in multiple shapes and sizes.
Benefits / Outcomes
- 75% reduction in the overall costs compared to traditional camera and lighting options
- Lower error rate and higher throughput of goods
- ToF camera eliminated the need for separate lighting and multi-camera system
- Efficient picking and placing without damaging bread or its packaging
Customers
KINE Robotics
Turku, Finland
Partner
Industry
Manufacturing
Solutions
About KINE Robotics
KINE Robotics Oy is a Finnish provider of custom turnkey robotics solutions. Its customers are in industries including food and beverage, logistics as well as semiconductor and general manufacturing. For more than 20 years it has designed, implemented and delivered fully customised robot-based solutions to fulfil unique client requirements.
The Challenge
The team at KINE was asked to develop a conveyor tracking system with on-the-fly robot picking of bags of bread loaves and rolls for an industrial bakery. Product sizes are irregular, and shapes differ from one type of bread to another, making it a challenge to determine the location and orientation of the bags on the conveyor. The transparent plastic packaging likewise makes it difficult to detect with optical sensors, due to low contrast and partial reflections.
Prior to the upgrade to a high-performance vision system, the process involved manual removal and depositing of bread packages, which was potentially error-prone and certainly not as efficient.
KINE sought to develop the vision system around Aurora Design Assistant software and a Basler Blaze 101 3D Time-of-Flight (ToF) camera as the sensor. The ToF camera was selected as it eliminated the need for separate lighting, and reduced concerns about contrast and reflections. “We really liked how smoothly the 3D ToF camera integrated with Aurora Design Assistant software,” confirms KINE’s project manager, Tero Urponen. “In other projects, we have had multi-camera systems that can get a bit cumbersome, whereas in this case, it was very nice to have one system to control everything instead of having to programme each camera separately.” Helping KINE through its development and acquisition process was OEM Finland Oy, a Zebra Registered Reseller. A member of one of the largest vendors of industrial components in northern Europe, OEM Finland Oy provides an exceptionally broad range of industrial automation solutions and electromechanical experience to machine and systems builders.
Basic implementation of the vision system took about one day. Testing, adjusting and fixing took about 80 hours, including time calibrating the robot. That’s a great turnaround.
The Solution
For this food packaging installation, the products being examined posed some serious challenges for the vision system: different types of breads have unique shapes, sizes and colours, and the semi-transparent packaging can obscure the contents, making it harder for the system to distinguish between products. A robotic solution with proper component selection would ensure a faster, more reliable and cost-effective manner of product detection and processing.
The inspection system is built around Aurora Design Assistant flowchart-based software and the Basler 3D ToF camera, which affords real-time measurement of up to 30 fps and provides precise 3D data to the vision software. The camera is triggered from a photodetector by way of a B&R PLC X20CP1585 programmable logic controller (PLC) communicating with the vision programme via a user datagram protocol (UDP). The PLC also synchronises the robot by similar means. An industrial panel PC from B&R serves as the platform to run the Aurora Design Assistant runtime environment and the associated vision programme. Handling the picking and placing is a Stäubli TS2-60 SCARA robot, which communicates via transmission control protocol (TCP) with Stäubli VALtrack software library for conveyor tracking and picking. The robot needed to be able to get a good grip on the plastic bags without causing any damage to the delicate bread or its packaging.
Camera data analysis is implemented using Aurora Design Assistant software, which also produces the central interface to the vision system. The 3D ToF camera records the surfaces as a point cloud with over 300,000 XYZ coordinates; Aurora Design Assistant converts the point cloud into a depth map, which is analysed using 2D vision tools to determine the grip points for the robot. The 2D tools provided by Aurora Design Assistant work particularly well on depth-map data and allowed the team at KINE to effectively solve their application.
Expressly designed to require no operator interaction, the vision system is located directly atop the incoming product conveyor, just before the robot working area, and functions completely autonomously.
The Zebra Difference: Outcome and Benefits
Overall, implementation of the vision system went smoothly for KINE. “Basic implementation of the vision system took about one day,” Sami Sinisalo, robot specialist at KINE, reports. “Testing, adjusting and fixing took about 80 hours, including time calibrating the robot. That’s a great turnaround.”
One challenge they faced involved issues caused by communication delays with the PLC trigger making synchronisation difficult. The ToF camera does not have a physical trigger input, and use of the panel PC meant KINE did not have access to the I/Os that would be available on a vision controller, as an example. “This means that the communication between the photodetector and the Aurora Design Assistant runtime involves a PLC and UDP communication,” Urponen reports. “The network communication does not occur in real time, which results in a variable delay.”
Another synchronisation challenge stemmed from interface of the Basler ToF camera to the Aurora Design Assistant runtime. The camera time varies from one acquisition to another by up to 50 ms, causing the product to move up to 20 mm (at 400 mm/s). KINE overcame this by using a trigger signal from the photodetector to determine the distance travelled on the conveyor; the same trigger is also used to trigger the ToF camera to commence image acquisition. The robot uses an encoder to track the conveyor movement after the trigger event, and the machine vision software then determines the transversal position and orientation before performing the analysis.
“Initially, there was a plan to use 3D information from the Basler ToF camera to determine the pick height for the robot,” Sinisalo notes. “We ultimately decided to create a recipe for each product with the software.” The transparent plastic bag is not visible to the ToF camera, and as the bread products within are arranged in random positions, the lack of a single, consistently flat surface makes it very difficult to determine the correct pick-height based on 3D depth-map information. KINE developed an identical vision system that only handles whole and sliced bread loaves. “In that case, the height measurement could work but we still believe that a recipe-based solution is more reliable,” Sinisalo adds.
Recipes are used to configure select flowchart steps so that a flowchart can inspect different product variants. This allowed for the use of a single project to inspect the different types of breads with their unique characteristics. Recipes can be created and configured in the vision software at both design and runtime.
Through use of the ToF camera and vision software, KINE saw a reduction in the overall costs related to machine vision by 75%, compared to traditional camera and lighting options. “Typically, there is a lot of trial and error with lights and camera settings to get proper contrasts for object detection,” Urponen notes. “The ToF camera requires no external lights, and the 3D information it supplies to Aurora Design Assistant is superior to any traditional colour/contrast-based detection technology.”
Conclusion
KINE and its customer are very pleased with the final vision system, with a lower error rate than the prior manual handling and with a higher throughput of goods. With the new vision system, the robot grip can handle between 25 and 30 packages per minute. “We’re already considering the possibility of adding some measurement limits for the products to ignore or flag faulty products as they move through the system,” says Urponen.
The team at KINE can see many opportunities for future integration of machine vision into its customed robot solutions, thanks in no small part to the success achieved with this 3D pick-and-place robot system.
