Robot picking

Once the machine is unloaded to the machine at one time, the monotonous tasks in batch part sorting and order fulfillment are labor intensive. It can also be life-threatening if parts or operations are heavy, sharp or otherwise dangerous. For many years, garbage picking robots have been dealing with these tedious tasks. However, there are so many applications that need to be implemented. The world of manufacturing and warehousing automation is watching anxiously.

Although more capable than ever, robot garbage collection still has its limitations. We are not sure about the Holy Grail – the random selection of robots. But there has been tremendous progress. With advanced vision technology, software and grip solutions, robots find their way in the unknown.

So why is the robot’s random trash can picking notorious? This is about accuracy. When robots are welcomed for repetitiveness, random picking requires confrontational accuracy. The robot must position the part in free space, and in an unstructured environment, the part maintains position and orientation as the part is removed from the bin. This requires robot dexterity, machine vision, software, computing power to compress the subtle balance between all data in real time, and to extract parts from the bin. A very high order, but not impossible.

The heat treatment machine tilt unit uses a 3D vision guided robot with a double end effector to position and pick randomly stacked car parts from a large bin. (Provided by Motion Controls Robotics, Inc.) David Dechow, an intelligent robot/machine vision staff engineer at FANUC USA in Rochester Hills, Michigan, said: “There is a lot of hype on the market, involving garbage collection and what it can do and what it can’t do. “The reality is that whether it’s inspection, 2D guidance or 3D guidance, machine vision is appropriate, reliable and reliable for a small part of the whole world of what we want to do. Judging from my years of practical experience, picking is now in the same category. There are real world abilities, but it is still a subset of the whole world of what we want to do. ”

Dechow works closely with the late visionary Adil Shafi in the field of vision-guided robotics and is considered an early advancement in trash cans. Many of his innovations still influence the continued development of these areas.

There are three main types of picking: structured, semi-structured, and random picking. The application complexity, cost and cycle time of each application are on the rise. (We’re here to distinguish between parts that are contained in a box or suitcase, not parts that don’t have a container. Let’s talk about why this is a key variable.)

Structured – Parts are positioned or stacked in an organized, predictable pattern in the bin so they can be easily photographed and picked.

Semi-Structure – Parts are located in the warehouse with a certain degree of organization and predictability to help with imaging and picking.

Random – Parts are completely random in the bin, including different orientations, overlaps, and even tangles, further complicating imaging and picking functions.

In these three subsets, special considerations apply depending on the characteristics of the selected parts or items and how they are presented in different places in the trash.

“Structured and semi-structured trash cans are usually easy to do, and they can implement most of the technology on the market quickly and easily,” Dechow said. “When we talk about the Holy Grail, I think of more random environments, overlaps, Entangled and multiple types of parts in a trash can. ”

He said that structured trash can picking can usually be done with 2D vision. “When you think about picking, everyone thinks about 3D imaging and 3D analysis. In fact, you can do a subset of the world by 2D imaging and 2D analysis.”

Success – geometric symmetry
Some features usually determine whether a part is suitable for picking. Dechow said they went from a “definite thing” to a challenging app.

Dechow said: “There is a fit in both the geometry and the performance of the random bin. “In the vast scale of the parts that guarantee absolute success, larger geometric non-complex parts, where the parts are barely relative to their random state of rest. difference. “No matter which part can be put down or lying in the box is one of its random rest states.”

“They don’t have a lot of strange features,” he continued. “They are usually not too heavy, and in all random breaks, they have enough planes that can be considered a successful choice. This 3D picking is today’s real-world success story.”

Dechow said that if the graphics or text of these projects are printed on it, as seen in the real world, it makes little difference. Most 3D image cloud generation is independent of the graphical markup on the object. (More orders fulfilled after this discussion.)

“It doesn’t have to be good, clean storage parts,” says Dechow. “Think of large cylindrical billets that are typically processed, forged or heat treated in a factory environment, possibly 20 or 30 pounds. But again, when they are in various static states and they only present a few geometric surfaces, it is Consistent geometry. It’s easy to pick up a robot and it’s easy to grab.”

This video demonstrates the picking of heavy steel billets in a trash can.

With the 3D area sensor to locate random parts in the trash can, the robot plans the next option. (provided by Motion Controls Robotics, Inc.) medium success – some complexity
From easy picking to more challenging garbage picking applications, these spectrums have more complex parts but are easily distinguishable features such as the (picture) ring portion.

The robotic random trash picking application uses the FANUC iRVision 3D area sensor and two robots to pick stamped car parts from the trash can and prefer a heat treatment machine. Motion Control Robot, RIA Certified Robot Integrator, designed and installed batteries from Tier 1 suppliers.

The Dustbin Picking FANUC robot is equipped with a double-head multi-function end effector that uses the Magswitch magnetic clamp to redistribute the parts in the bin for easy disassembly and assembly. The SCHUNK double-finger clamp can be removed and placed in the chute. The sensor on the chute passes the direction of each component to the controller of the second robot, indicating whether the flange is facing up or down, so the robot knows how to grab the part. The robot then picks up the part from the chute in the correct direction and inserts it into the heat treatment machine. The same robot picks up the finished product and places it on the outsourced conveyor for transport to the next operation.

Before reaching the current solution, the integrator reported experimenting with a variety of end-to-end tool designs and visual processes. Unusual random picking app. 3D technology still requires skilled integration, and the more complex the components, the more difficult it is to integrate tasks.

FANUC’s Dechow explains that it is a more complex structure in this intermediate frequency range.

“Imagine an object like a crankshaft, usually circular, but very long, and with many geometries. It may appear as a cylinder from one end. From the other end, it may be a small diameter rod. This is A very long object, so it may overlap with other parts. When another part is at the top, the 30-pound part suddenly becomes a 50 or 60-pound part.

“These types of parts have a complete set that you can find in heavy industry and factory automation production,” he continues. “These types of parts are usually delivered in boxes. You will see a part of the top surface of the assembly, where the image is taken, and the rest is like an iceberg. Most of it is below the surface, so it’s hard to understand Its optimal clamping technology.”

Still challenging – packaging, packaging, deformation
Then even with 3D imaging technology, there are components that are particularly difficult to image with machine vision.

Dechow explains: “For example, these parts are thin but potentially wide and wide parts that are intertwined and superimposed on each other. “These parts make it difficult to distinguish the parts because the height changes so thinly, the parts The geometry makes the scene very confusing.

Vision Show – May 3-5, 2016 – Haydn Convention Center – Boston, MA “Packaged parts, plastic bag parts or soft (deformable) parts.

He added: “This part is very difficult to distinguish in the garbage picking application.” “No one will say ‘no’ attitude. “We want to push technology to the limit as much as possible, and we do it often.”

Random Bin with 2D Vision

One company that continues to launch signals is JR Automation Technologies, LLC, of the Netherlands, Michigan. The RIA-certified robot integrator is one of the small test sites for FANUC vision-guided robotics.

Tyler McCoy, JR Automation Control Engineering Manager, said: “We will feel new technology during the testing phase, and then we will be asked to evaluate and provide feedback. “We have been doing this for so many years. We are the test bench for the original FANUC iRVision. ”

McCoy said they integrated a large number of structured and semi-structured garbage picking applications and regarded them as mainstream. However, random garbage picking devices are still few and far away.

McCoy said: “We recently implemented a random picking solution for the car seat back assembly battery. “This involves a main U-shaped frame that looks like a seat back and two tubes are welded to the headrest. On the frame, then another part called the decorative frame bracket, which extends across the middle of the frame to sit on the seat cushion of the car.

“This is an assembled and welded battery,” he explained. “Our task is to take all of these parts out of the batch, assemble them into welding fixtures and place them on a four-position indexing table. Then we have two robots that weld the components together.”

He said that the main U-shaped frame was loaded onto the gravity frame to facilitate easy feeding of the welding process. The two headrest tubes are small enough to be used in bowl feeders.

“But this decorative frame bracket is about 16 inches long, 9 inches wide, and about 0.75 inches deep. This is this long sheet metal stamping. There is no really good feeding technique available,” McCoy said. “So we evaluated the 3D area scan, but we found that the low-profile geometry of the part made it difficult to get good contrast or good 3D features in the pile. The 3D pickup of this 3D application has not yet been completed.” (This will belong Thin, banded tones with a larger spectral range).

McCoy said: “First, the best thing to do is to choose garbage picking, starting with evaluating parts.” In this application, we found that these low-key parts are often locked together. We also found that this part can naturally sit in some direction of the warehouse, limiting our ability to target picking strategies based on part benchmarks.

“So we implemented a solution where we have a robot that takes the tray out of the box. Using a non-contact displacement sensor to detect the height of the bin, we create a virtual topographic map. We pick up three or four parts at a time. Use a compliant end tool with a magnet to pick from a high point to a low area and monitor as the part falls. These parts are placed in a flexible loop feed system that uses a series of conveyors The parts are split, then the 2D camera and the second robot pick up the parts. By combining our own software, compatible arm end tools and some general picking strategies, we can take the parts out of the box and reacquire them. Split it into more traditional flexible feed applications.”

The battery contains a total of six robots. One robot performs garbage collection, the other pulls a single bracket from the flexible conveyor, and the third robot picks up the main frame and the headrest tube from the bowl feeder, the two welding robots, and the unloading robot.

McCoy said: “The current system provides our customers with a 2-1 / 2 hour buffer for each bin on a 20-second cycle machine. “We did this system with two bins, so They don’t need to wait for the bin to cancel it. The robot will empty a trash can and then 2/2 hours before the robot 2 to cancel the trash can No. 1. ”

The robotic random trash can picking and part loading system uses a 3D vision guided robot with magnetic grips to position and pick up the components for the heat treatment operation. (provided by Midwest Engineered Systems Inc.) The seat back components must be manually loaded into the welding fixture prior to the JR automated process. McCoy said this is the second system they have integrated for this customer. The first system has been in production for about six months.

Sensor, software and EOAT
Random picking requires the convergence of technologies, especially the three main components of robotic intelligence: sensors, software and end-to-end tools. The development of all three areas has brought us closer to elusive awards.

“One of them is hardware,” McCoy said. “It just continues to get more cost-effective, higher resolution and faster processing times. The ability to generate 3D point data is becoming more and more industrial. Companies like Keyence and LMI are releasing very interesting products. It seems that everyone is trying to bring 3D point clouds into the industrial arena.

“What’s more important is the software component,” he continued. “There are a lot of cool companies that are independent of robot manufacturers, trying to say that this is just a software competition. You have a company like recognizing robotics that is using 2D cameras to make a very cool six-degree-of-freedom system” (we cover recognition Robotic random picking system intelligent robot: a feast of feelings.

McCoy said: “I think all hardware manufacturers have made great progress in the resolution of 3D point cloud generation hardware. “There are many smart software developers who have used it. I think we are at the point where these two technologies converge at the right time. I am very happy to see that there will be there by 2020. “ RIA-certified robot integrator Midwest Engineering System

FANUC’s interference avoidance function is software based and is standard on 3D area sensors. Other robot manufacturers and software developers have similar algorithms, often referred to as obstacle avoidance.

“You can model the robot’s end fittings, you can model the mold at the end of the robot and model any model in the gripping area that might interfere with the task,” explains Dechow. “As the robot is sent to a completely unknown random point, the software calculates the desired target and changes the method and pinch. Like a part that leans against the wall of the trash can, or a part at a certain angle, the robot must take it Drive to the wall to get the parts. This is a very elegant side of the pick, not many people think.”

AI picks Bin
For a software developer, this is all their ideas. Laboratory researchers born at NASA are preparing humanoid robots for unknowns in space exploration, and general-purpose robots have been performing tasks. They are fighting for accuracy.

Peters said: “In the past few years, the biggest thing that happened in technology was that we were 100% accurate in trading. “Under the hood, the algorithm is getting more and more complicated. Learning functions are becoming more and more extensive. All of these things around artificial intelligence are constantly expanding, based on applications. If you consider how we started with mobile boxes, we have already entered the trash can, and now garbage collection has begun to expand to where we can handle thousands of SKUs. ”

Universal Robot” Random Trash Purchasing Software Solution The modular application platform has two components: traditional vision and communication between spatial vision processing devices, sensor data and drive control of robot functions and calibration. The new cortex will be its real-time machine Learning or artificial intelligence is combined with 3D visual results generated by spatial vision.

“We are using parallel processing,” Peters said. “This gives us the ability to take large data sets, analyze their contents in milliseconds, understand what is happening, and then drive the behavior in the machine to whatever is needed for the transaction. We are in a supply chain The point of competition with human speed.”

Order fulfillment
A supply chain solution involves a partnership between Universal Robotics, R/X Automation Solutions and Yaskawa Motoman, providing a comprehensive solution for robotic pharmacy order fulfillment systems.

“We put Neocortex on some industrial robots for order filling, until this requires manual operation. We have established triple redundancy in the system, absolutely 100% guaranteed to be selected for the ordered drug.”

This video demonstration of random garbage collection for pharmaceutical order fulfillment shows the speed at which the stacked items are selected, regardless of shape, size, orientation and location.

Peters explained: “R/X Automation’s customers are large pharmaceutical companies that are directly customer-facing.” In their transactions, they provide the entire 100% certified automation solution that customers need and then process the work unit components. Historically, this is where people are in the supply chain. We provide turnkey solutions for this specific unit, including start-up, industrial robots, sensors, controls, processing and the safety required for the machine to operate in this unit of work. ”

“We load the robot, CAD model into spatial vision, which automatically learns how the machine starts and moves through the space. (Like the end effector) and then uses it as part of the driver to make obstacles about how to do it. decision.”

E-commerce and order fulfillment are an important growth area for robotic garbage collection.

Peters said: “Wal-Mart, Costco, Kmart, Target, they all have online ordering.” “Until now, these tasks require a person, once it is in the packaging station. The process of packaging the shipping box is a very good process for picking garbage. Applications.”

He said that the item can be any geometry. But picking up a bottle of detergent and a batch of lipstick is very different. Then the challenge will become the terminal executor.

Grab the challenge
Many people believe that smart grip technology is the last challenge we have to overcome, turning random picking into mainstream. FANUC’s Dechow saw great potential to seize the solution.

“I found that we can extract the required data from 3D or 2D scenes multiple times, but the question is how do I get something into the bin and the object to successfully receive it given the available data?

“The problem is that a clip usually doesn’t solve every random quiescent state of the parts in the bin,” says Dechow. “Of course, the imaging component is critical to determining which components can be imaged, but the end effector becomes one of the last components on the critical path to determine if the part can be selected.”

The soft start fixture uses 3D vision-guided robotics to pick fresh produce from the bin. Adaptive clamping technology (provided by Soft Robotics Inc.), such as soft robotic flexible grippers ready to meet the challenge.

Peters said that just as they can dial traditional terminal effects in CAD, they can do the same for these dirty clips.

He said: “You know its inherent capabilities and the range of variables that each activated finger moves. “You can also set these variables when you put them in CAD. ”

Now, check out this video in the Electronic Adhesive Robot Grabber Pickup Box from the Box and Shelf Compartment.

Various companies are working hard to implement application standardization picking techniques for a variety of industries and orders. This makes it easier and more cost-effective for small and medium-sized companies to implement robotic random bin collection. This video shows some off-the-shelf solutions.

Even Google’s moonshot chaser is taking action. Observe a robotic team with adaptive fixtures and use in-depth learning to pick random e-commerce projects