If you want to teach software to recognise objects in images, you don’t have to start from scratch. There are large, freely available datasets that can be used to train such systems. Prof. Florian Weininger lists what they contain: cats, cars, traffic lights. Granite paving stones do not feature. “Quite simply, nobody has been interested in them so far,” he says.

You might dismiss this as a technical aside. Weininger is a professor at OTH Regensburg, where he heads the Walz 4.0 project. He means it in a more fundamental sense: “When it comes to digitalisation in the construction industry, almost everything revolves around planning and AI. Everything is supposed to be planned more quickly. But we’re the ones sitting in front of the computers; the people on the building site aren’t.” Those who lay paving work on the ground, and this is the case for many in the building trades. “These people are simply not being taken into account enough when it comes to digitalisation,” says Prof. Weininger.

Walz 4.0 aims to change that, at least on a small scale. The name alludes to the traditional journeyman’s travels. In this Interreg project, universities and craft organisations from Germany, Austria, Switzerland and Liechtenstein are working together. OTH Regensburg is addressing the question of how digital technology can be integrated into craft work, specifically through initial and continuing training. “At its core, it is a teaching project,” says Prof. Weininger. The paving demonstrator brings together four key elements for this purpose: artificial intelligence combined with augmented or mixed reality, exoskeletons, robotics and sustainability.

These four themes fit onto a Euro pallet. On top of this stands a box containing a bed of sand and natural paving stones, and four QR codes at the corners mark out the area. A standard webcam is pointed at the area, along with an ordinary laptop. This is where the plan is stored: a digital laying pattern – perhaps an arc – in which each stone has a number, a position and an orientation. The camera films what is actually laid out in the sand. An AI recognises each paving stone and compares it with the plan. The person laying the paving can see the result on a tablet or through mixed-reality glasses, which superimpose the planned pattern onto their field of vision. If a stone is out of place, the system alerts them that it needs to be moved slightly. A digital assistant called WALZI translates this into feedback that doesn’t require a degree in computer science. “The recognition itself isn’t rocket science; there are plenty of solutions for that,” says Prof. Weininger. “The real work lies in the implementation.”

The box combines four components. Two of these are the AI-supported placement aid and the mixed reality feature, which shows where the next stone belongs. The other two are designed to support the body. An exoskeleton supports the back and arms, delaying the onset of fatigue when working in a kneeling position. And Sid, a robot dog, provides robotic assistance on the building site: he can carry materials, such as paving stones. Both are being tested not only to see if they reduce physical strain, but also to assess whether they get in the way whilst working.

Granite comes into play during implementation. “A granite paving stone doesn’t have clean, sharp edges,” says Prof. Weininger. “The algorithms struggle with that.” In November 2025, Master’s students in civil engineering generated the missing training data themselves. There were 18 participants in the course ‘AI Technologies in Construction’, and their internal motto was: ‘We’re building a WALZ AI’. They collected images, labelled them, trained and tested models, including using artificially generated data. The result was several individual models and a shared dataset.

The fact that the system works with natural stone rather than standardised concrete blocks is linked to the circular economy. Natural stone is durable and can be reused. However, every stone is shaped differently, and laying it neatly requires experience. The thinking behind the project was that if software could recognise how an irregular stone fits and where it belongs, even less experienced workers could work with such material. This would reduce reliance on industrially standardised paving stones.

In the project documentation, the sessions are called ‘The Paving Box on the Road’. This time, it is the box itself that is on the move. In April, it was housed in the construction hall of the Training Centre of the Lower Bavaria-Upper Palatinate Chamber of Crafts in Regensburg. There, trainee road builders are learning how to lay paving in a large training workshop. Around 15 of them tried out the box, along with some 40 participants from other trades. A photo from the Chamber shows trainer Ulrich Liebl kneeling in front of the paving stones with his glasses perched on his head.

In early September, around 30 new Strabag apprentices tested the box at the Wetzendorf Training Centre in Nuremberg. There, for the first time, the mixed-reality application was no longer restricted to the area of the box. QR codes can now be used to map out any open space and display digital content on it.

“We’re constantly on the move with the box,” says Prof. Weininger. The feedback has been consistently very positive. “People feel seen. The reaction is often: ‘This is technology designed for us.’” Many were also amazed that you don’t need a tech giant to make this happen. The devices are available to buy; you just need to adapt them for the specific purpose.

There is some criticism, too. Some people find they break out in a sweat when wearing the glasses. The HoloLens, which the team uses, has a transparent visor, much like a head-up display in a car. However, this doesn’t work in sunlight. That’s why the team is also trialling a pair of glasses from Meta that work differently: There’s a screen positioned between the eye and the surroundings. Built-in cameras film what lies ahead, and the screen displays this image live, together with the superimposed 3D models. Sunlight isn’t a problem with this setup. On the other hand, some people find it strange to see reality only through a screen.

For the team, it’s all about the data. Prof. Weininger lists what interests him: “Do the glasses get in the way whilst working? Do you sweat under them? What happens in unfavourable sunlight? Or do we ultimately need a completely different technology?” With the exoskeleton, the question is similarly practical: “An exoskeleton might be designed to drill holes in the ceiling. Does it also work when you’re working in a bent-over position?”

To ensure the answers aren’t based solely on gut feeling, the Berlin Institute for Social Research (BIS) is monitoring the tests. In March, tradespeople from two paving companies took part in a standardised study at the Building Lab at OTH Regensburg. They had varying lengths of experience in the trade. They carried out a reference task without technical assistance, as well as scenarios involving AI-assisted positioning, mixed-reality glasses, an exoskeleton and a robot dog. Sensors recorded movements and muscle activity, supplemented by interviews and observations. The study examines acceptance and usability, physical strain, and whether such technology fits into a typical working day.

From a technical perspective, there has been some progress. In a laboratory test involving experienced paving specialists, the system provided reliable positional feedback for most paving stones, with tolerances in the centimetre range. This is stated in a paper presented in July by the team led by research assistant Simon Höng at a European conference on digital construction in Corfu. In the paper, paving is cited as an example of a craft that has so far largely been overlooked by digitalisation.

The authors also highlight what does not yet work. When individual paving stones are rotated, the results are not yet conclusive. If a stone is obscured, the system loses track of it. It cannot determine whether the stone is positioned too high or too low. Dirt and reflections on the QR codes interfere with the calibration. On a real construction site, it is precisely these weaknesses that could become a problem.

There is a plan in place to address most of these weaknesses. In future, the system is to determine the rotation based on more reliable features that indicate how a block is oriented. To deal with concealed blocks, several cameras will be used to capture the area from different angles, so that a block not visible to one camera is detected by another. Height is to be measured by a LiDAR sensor, which uses laser light to determine distances and thus also captures the third dimension. The sensitive QR codes are to be replaced by more robust, raised markings, which are less affected by dirt and reflections. Furthermore, the system is designed to check the entire pattern rather than just the most recently laid brick. A larger study will reveal how well all this works. If successful, applications beyond the training workshop would also be conceivable: tolerance checks, documentation of construction progress, invoicing and materials logistics.

As for the exoskeleton, it remains to be seen which movements it actually alleviates, how well it can be adapted to different people, and whether it is comfortable to wear over extended periods, as experience shows that if the system is uncomfortable, it will not be used. As for the dog robot Sid, its use is currently limited primarily by its load capacity and the fact that it still lacks a gripper. For use on a building site, it also requires safe routes and reliable cooperation with workers. The next stages of development will therefore focus on the entire ‘Pflastern 4.0’ system.

The next pilot is due to be ready by the end of the year, again on a pallet, this time for drywall construction. It involves a curved surface. “You can’t simply press a rectangle onto a curved surface,” says Prof. Weininger. That is why it is broken down into triangles. An app will allow tradespeople to move the points and edges of the frame. Every adjustment results in a different surface and a different bill of materials. Cutting is carried out using augmented reality – the system is designed to recognise the parts without a QR code – and assembly is carried out using smart glasses and an exoskeleton. This shifts part of the planning process to those who will later be handling the panels.

Prof. Dr Thomas Linner took the lead in submitting the application on behalf of OTH Regensburg. He is responsible for the key areas of construction robotics, automation and human-centred assistance systems. Prof. Florian Weininger heads the project at OTH and is responsible for its transfer to the skilled trades, vocational training and continuing professional development. Prof. Dr Charlotte Thiel contributes her expertise in building materials, durability and circular construction. Prof. Dr Mathias Obergrießer contributes expertise in Building Information Modelling, parametric modelling and artificial intelligence. As research assistants, Anne-Sophie Saffert is working on human-centred assistance systems and ergonomics in the construction industry, Simon Höng on AI-supported stone recognition and its integration into teaching, and Lukas Esper on sustainability and the circular economy. The Berlin Institute for Social Research is providing social science support for the trials. Walz 4.0 is funded by the EU, Switzerland and Liechtenstein as part of the Interreg VI Alpine Rhine–Lake Constance–Upper Rhine programme and will run until July 2028. The lead partner is HTWG Konstanz.