Winning innovation soon in a billion mobile phones

It began in 2005 when Harald Klomp and Jakob Sandström were doing their degree projects at the Centre for Image Analysis at Uppsala University. In collaboration with the Swedish University of Agricultural Sciences in Umeå and a company that built drones, they developed an algorithm for making maps based on video images, for use in forestry and agriculture.

“This posed numerous challenges in terms of graphics and algorithms. The algorithm was developed for image stabilisation and mosaics, so as to superimpose the frames correctly to agree with reality,” says Klomp, currently a business strategist at Vattenfall.

After the degree project ended, they continued to work on that project, submitted patent applications and eventually obtained several patents on their original innovation.

Continued working on drone video

They belonged to one of the first cohorts at Uppsala Innovation Centre, obtained a business coach and made contact with a business angels network in Stockholm. The company was founded in 2007 and the first business plan was to continue working on drone video, though now in a military environment.

Film from drones was used to support decisions in war situations, both in Iraq and in Afghanistan, and to begin with the technology was developed for use in war zones.

“This algorithm had the advantage of being unusually innovative and slimmed down. Many video stabilisation methods are sharp enough, but incredibly calculation-heavy. This was supposed to run in real time on a normal laptop, which demanded a tremendous amount of optimisation,” explains Andreas Lifvendahl, CEO of Imint.

Applications in industry

Lifvendahl joined the company in 2010, when it was three years old and had begun to look broadly at applications in industry. One of the early projects involved the Swedish Navy’s submarine rescue craft, which was going to be equipped with better cameras but invested in software instead. Saab was another early client, developing technology for remote-controlled air traffic control towers for small airports.

“We also delivered software to them, but those sales processes are pretty long and being a small startup with risk capital, after five or six years the owners became rather restless. It was simply taking too long.”

Adapting the technology to smartphones

At some time in 2013–2014, they therefore started to adapt the technology for smartphones. This was a couple of years after Apple had released its iPhone and the whole ‘smartphone race’ began to get going, Lifvendahl notes.

“It very quickly evolved from a smartphone race into a camera race. We really had to go back to the beginning, we dismantled all the original structure step by step. One major point was that smartphones had very good high-precision movement sensors and if you can use this data for video stabilisation, it makes things vastly simpler and much more energy-efficient.”

Much of their current technology is based on combining data from the camera sensor with other sensors in the phone, and incorporating this in the software embedded in the phone. Their software can now be used in all types of Android phones.

“We timed our entry into the mobile market just right and were also very good at the real time aspect. It’s difficult to do all the calculations required before the next frame comes. We only have a few milliseconds to do it. It’s a matter of interaction, the hardware in the form of memory and gyro sensors has to work in combination with a smart mathematical algorithm that does the job,” says Johan Svensson, Imint’s CTO.

Clients in Japan, Taiwan and China

Imint now has 48 employees in Uppsala and clients primarily in places with Android manufacturers – Japan, Taiwan and China. It was hard work getting established in 2016–2018, Svensson recalls, who was new to the company at the time. The first client he worked with was the Chinese giant Huawei.

“It was exciting that things were happening, but at the same time we realised that we didn’t have everything in place that a big company demands in terms of test procedures, test environments, security, policies and so on. So there were a lot of weekends and late nights... But of course you learn a lot too from this kind of journey.”

Looking ahead, they plan to pursue two tracks. They will continue in the consumer market, but will also resume work on decision-making support using video technology.

“One of the markets we’re working in is real time video in industry and healthcare, with body-worn cameras. This can range from home healthcare to ambulance healthcare and clinics, with remote help from an expert,” says Lifvendahl.

Many degree project students enlisted

They still have a good deal of contact with Uppsala University, above all through degree projects. Each semester four new degree project students join the company, from the Centre for Image Analysis, the Computer Science programmes and in recent years the Programme in Sociotechnical Systems Engineering.

“It’s a way for us to get a broader handle on our users. How different products are used, with a particular focus on future markets and solutions,” says Svensson, who is often supervisor.

What do you get out of it?
“Above all, it’s a good recruitment path for both sides. They have a chance to see what sort of a company it is and we have a chance to see who they are. It’s also a good way to try things out. It’s unusual for a degree project to lead directly to some form of product, but it stays there on the shelf and a year or two later you see something there that has potential for further development,” Lifvendahl says.

So degree projects are still valuable for Imint’s development, just as when it all started once upon a time.