Impressive Microsoft Application: Unwrap Mosaics Will Amaze You

Ripten writes:

"The reaction I had when I saw this video could easily be summarized as a holy shit moment. Microsoft's Unwrap Mosaics is an amazing application with the ability to fully translate video into 3D model surfaces through the use of 2D-to-2D transformation, texture maps, and object-space occlusion masks. Sound complicated? This is something you definitely have to see to believe."

[InMyOpinion]

Looks like MS is poking a finger in the eye to all those critics who say they never invent anything on their own.

@ MaximusPrime who is reporting the article - He does not have to edit anything out since he is just quoting the author.

[v1c1ous]

but he can still s***

[Nostradavis]

sorry guys. i fell asleep and woke up to see this posted. didn't have time to remove the word poo poo.

[Sheddi]

I like this!
this is what i want big company's to do!
with Microsoft owning billions of dollars they should invent stuff!

[marinelife9]

That's amazing!

[The Lazy One]

Watch a CES keynote once in a while. Some of the stuff you see will amaze you. (Although they may not do them anymore... :( )

They have huge RnD departments that you barely ever see if you don't look for them.

[JasonPC360PS3Wii]

That creeped me out and fascinated me at the same time. It's like looking at a car accident, it's horrible but you can't look away.

[meepmoopmeep]

that's really cool tech.

[predator]

now that is cool

[titntin]

I can't imagine that this is of much interest to that many people as its got absolutely nothing to do with games at all.

For a guy who occasionally does film and advert post processing, like me, its pretty interesting though.

In terms of its 'inventiveness', mosaics are fundamentally well understood, whats new here is successfully 'unwrapping them - though its notable that this only worked is certain circumstances. Its still a nice development of an existing tool set though, and just one tool to add to arsenal for those occasions where it going be the best choice for the job.

[titntin]

Lol at the disagrees from gamer fanboys with absolutely no knowledge of the post industry or process at all!
The disagrees are because I didnt' bow and go - "wow I've not seen that before?" People on this site really are pathetic...I wander why I bother to try and have a rational conversation on here at all...

[Twilightx7]

As a 3D artist, I have to inform you that you're quite wrong about it having no usefulness in the gaming industry. Those of us who model people and places for current software often need to have a level of photorealism in order for the assets to be acceptable for production, and to successfully unwrap footage of someone's body, or a face, for an artist to then manipulate and apply to an object/3d character, saves us an ENORMOUS amount of time. ILM has software similar to this, although instead of unwrapping it simply creates the models based on hundreds of photos at every angle.

So sure, it doesn't do much for the average gamer or hobbyist, but it affects how the games they play are made.

[Xi]

Specially on cinematics in videogames, you can take the model animate it then apply a series of post processing effects to a flattened image, this is extremely versitile. Being able to quickly add more extra ships to scene that look like they belong there without requiring modeling or a skybox. Adding more detail on characters faces etc.

I think you're just spouting smoke, you probably do absolutely nothing with regards to what you say you do. First it was you worked as a concept artist now for film and advertising.

[titntin]

What are you talking about mate? You can do what you are saying in ANY of the 2d post effects apps available - 'shake' being an industry fave or After effects on PC or Nuke or many other post processing packages that already do that job.
I've never claimed to be a concept artist - ever. I'm a games artist and animator, and have been for over 15 years, and do just about everything EXCEPT for concept art work, but specialise in animation. I had occasion to do a big and sucessful add campaign last year involving lots of robots and digital set extensions (the Carling C2 advert) and as a result got hired to do a whole bunch of post process work on a forthcoming sci-fi film. I prefer to make games, but I'll take the contract thats on offer when I have to..

@Twilightx7 Its obvious you don't do this for a living or you would recognise that there are tons of tools available already that will unwrap a 3d object and apply photographic textures - the tech to do this has been in use more than half a decade and is routinely used on every single game I have worked on for many years.
The whole point of this particular tool, is its ability to track 3D motion from a 2D image and apply this to a 2d tracked scene whilst including 3D occlusion data. This is cool, but there several solutions for this, though more are always welcome. If all you saw was a simple unwrapped photo texture, then you simply did not comprehend what you were watching at all. No professional artist anywhere would use this tool to create a texture for a 3D model - there are far better solutions for that task.

[The Lazy One]

There are programs that do it, but to say this won't be helpful is dumb. The fact of the matter is that this does it better. You don't need to have multiple pictures, and you don't have to do anything to the pictures to set it up right. You just need a video. That is a lot easier than most of the stuff available.

Also, it could do a lot for generating 3d meshes quickly based off of a video of a building once they take the warp mesh code up a notch.

There is stuff that does it, but this looks a lot easier, and that's never bad.

[titntin]

Thats not true at all mate. Look at the image it produced. If you do 3d texture mapping at all, you'll know that its just not a good enough unwrap to be a substitute for a proper unwrapped 3D texture.There are loads of contours that are not unwrapped at all and the results are skewed and would be very difficult to effectively map to a 3d model.
Its coolness and relevance is ONLY applicable to editing the footage from where it was derived.
Just to explain one on the many problems with using this kind of result on a 3d surface take a look at the unwrap used as the image at the top of the post. The nose is the most obvious example so we'll look there. There is no pixel detail for the Right hand side or underside of the nose. Any attempt to use this as a texture for a 3D model would result in 'streaking' in these areas as there are no pixels to work with. As a result of the fact that the unwrap is not proper, texel density across all the contours would be uneven - the very thing you are trying to avoid when unwrapping and mapping.

I do understand the confusion here - the image produced does resemble what you are trying to achieve when you unwrap a 3D object for mapping. But its not. Its relevance is strictly limited to the footage from where it is derived.
Once again, it is a truly excellent tool. I'm not knocking it, and I'll look forward to getting my hands on it. But its a post process tool only and its composite layer results are NOT suitable for 3D mapping - at least not with out some further significant reworking in photoshop.
I'm not here to score 'brownie points' from a bunch of game frothers, but I occasionally meet some good people here and have a good debate. If I make a point about a technical matter like this, its because I'm hoping my knowledge will enlighten others.

If you still wish to claim this does something it doesn't even claim to, when its blatantly obvious that its not actually a good tool for that, then that is of course your prerogative.
However, my points are not 'dumb', they are the realistic appraisal of someone who has spent an entire adult career doing this stuff, and my opinion is offered here to help people understand what they see, not to simply rebut the words of someone I neither know or particularly care about.

[dodgecity]

Tin tin, you obviously haven't a clue as to what you're talking about. Lol, the more you are proved wrong, the more snooty you get. We don't want your *bulllshit* expertise here.

[titntin]

This coming from a guy who's hardly been on the site and never made any constructive contribution to the site, ever?

Fortunately, not every user on here is as totaly pathetic as you, and I have an entire PM inbox of mails from other dev's on the site and users alike, thanking me for the insightful posts I make, and urging me not to take any notice of the blithering idiots like you.

How can a brain dead 'Gamestop' sales person know even the first thing about 3D mapping? You don't. What we don't want is yet another teen age know nothing making nothing but offensive posts and not talking meaningfully about anything at all. GTFO.

[The Lazy One]

I'm not saying it's perfect or that this exact thing is what would be needed. I'm talking about using it as a base platform to launch off of.

There are definitely some impressive things about the demo. I know it's lacking some major quality, but consider the possibility of the quality you could get with multiple passes over an object for importing 3d models and the textures for them. You could get an easily passable fully 3d model of a building by walking around it with a handheld camera 2-3 times. You'd have to touch up a bit definitely, and the tech is nowhere near that point for this software yet.

Just saying it's a cool thing to keep your eyes on. Imagine some game environments where the developers can actually take in the 3d and texture data for even a small part of a city. It would freak the balls out of me if I was playing a game and ran by a starbucks I frequent.

[DaKid]

Thats really cool, I didn't even know they were working on something like this.

[BIGELLOW]

...but Microsoft did not invent this editing technique, nor is this the only software that can do this. All Microsoft is doing is taking existing software (that is too cost prohibitive for home users to enjoy) and are making lesser version for people at home to get to play with.

Given this, what did Microsoft "invent" this time?

(I'm not bashing Microsoft... this is some pretty neat stuff... I'm also looking forward to seeing MS Surface in some places, even if it only turns out to be a gimmick... gimmicks are fun, after all. I just question your decision to suggest that this is something Microsoft has "invented" as opposed to simply throwing their hat into an already existing ring.)

[Nostradavis]

good point. bubbles.

[Lumbo]

welcome to 2006
http://www.youtube.com/watc...

[RealityCheck]

Good one Lumbo, actually the video says 2005 : )

[doshey]

ya but as usually someone everyone thinks ms came out with it when the same thing is going on in that video oh look at all the disagrees guess the truth is too hard for some people

[KingME]

Do you folks ever quit. Who gives a sh!t who created the technology, it cool tech. Rather than seeing it for what it is, you're more interested in proving that MS didn't invent it.

You really need to find something better to do with your time.

[doshey]

its the fact that everyone gives sony crap for the stuff they do but no ms is allowed to take everthing that has been already done and say its there idea and dont get crap for it

[Sitdown]

You must not be on this site to often....well at least around this years E3. Cause Microsoft has gotten the full grunt from the community with regards to their avatars.....obtaining the next Final Fantasy outside of Japan, the Netflix deal......and the list goes on.

[KingME]

How did this article become an issue between microsoft and sony. Is there any lines not crossed by the modern day fanboy?

[HardcoreGamer]

apparently it was shown by sony and toshiba lol;. i was like ehh, why are ms gettin praised when its been done already by sony and co.??

anyway that aside 360 has gained ground in japan, but its been 3 years isnt it too late for it to do any good in japan.

360 has deeeeeeeeeeeeeeeeeeeeeeeeeeeee eeeeeep pockets ppl

[BigKev45]

With this app, I guess you can make your johnson look bigger.

[DaKid]

They should get this on the market fast, before I go to eharmony.com.

[Elven6]

That was pretty amazing!

[Pain]

M$ cant and never have had a Innovative idea ever...unless Copying is a skill.

[GiantEnemyCrab]

Yes, who did they copy? Why don't you provide some backing of your comments instead of just trolling in, saying stupid things and then never replying.

[Gamingisfornerds]

...how exactly?

Lame.

[Bzone24]

...how exactly?

Lame.

[Kyur4ThePain]

He asked a question related to the article posted here.
Not necessarily beneficial to anyone, but at least on topic.

[Gamingisfornerds]

Your reply to my comment benefits anyone...

...how exactly?

Lame.

[Elven6]

It could be implemented into gaming somehow, maybe getting a character model from a dead person through video footage?.....

[Mr Bubbles]

It's true that this is an elaboration of an existing tool set, but I think it's a little disingenuous to talk about Microsoft (or Sony or any other large company) as though it were a person. I'm not specifically accusing anyone in particular as we all (myself included)do it from time to time. I just think that because of the tendency on forums to debate consoles as entities, we easily lose sight of the thousands of individuals and unrelated factions within these large companies.

I also think it's shortsighted to say this has nothing to do with gaming. Creative spirits draw inspiration from many places. The tech that fuels eyetoy has its roots in cognitive science experiments in edge, color and object detection in human vision. I guess vehement nay sayers at the very least exist to be proven wrong.

[titntin]

Well written mate, and while I DO agree that games creators draw their inspiration from all places, thats a tenuous link at best, and means I could post a story on some great modern architecture - as it would be inspiration for many environmnet artists and digital set designers everywhere.

The fact is, I do actually find this interesting, as it relates to my area of work, but its relation to games is negligable, despite the misinformed posts of those who obviously don't even understnd what they are seeing.

[Mr Bubbles]

That's a valid point. Perhaps when I think of gaming, I look at it from a broader perspective, including the industry and emerging technologies that inform it. For instance, the "photosynth" software that's been around for a while now takes any and all images that are found on the internet or other databases and finds patterns and overlaps in order to create scalable 3 dimensional maps of the world and its objects.

http://www.ted.com/index.ph...

Countless pictures of Notre Dame cathedral in Paris, fannypack wearing tourists and all, are readily available for the software to process and generate a 3d map- it can then be tagged with xml metadata and additional information and links if desired. Imagine if the data could be used to model game worlds, or if changes in the real world resulted in automated changes to the architecture of the game world; a 3d wiki of sorts. I don't know if it's a good idea, but it seems like fertile ground.

The unwrap mosaic tech may not have as direct of a potential link, but I can see how some of the ideas could evolve into psEye or other cam based gaming applications. I would think that any optimizations in software for 2d video to 3d maps are welcome and relevant to the developers that work on these these types of games.

Your point is well taken though.

[titntin]

Nice - thanks for the link, I hadn't seen that and its very intersting.
Thanks for a reasoned conversation too - something becoming increasingly rare on the net these days. Bubbles for Mr Bubbles! :)

[Fishy Fingers]

@Mr Bubbles

Thank you for that link, that is simply amazing. +1 for you bro.

[Lets-Game]

thats a great invention, i can see my self using it. hope its out amd ready soon.

good job ms.

[zonetrooper5]

Anyone seen the Surface by Microsoft, looks so awesome but its a shame it will take a few years to get the price down really would love one in the living room.

[PenisaurisDix]

that just milks windows and office.

Tech like this helps them with this goal . I say keep up the good work.

[Nostradavis]

I agree Nancy. Sorry for your loss.

[IaMs12]

Wow that is amazing.... that would be a great addition for movies... make some awesome last minute changes to anything...

[N00BZSUCKASS]

I look forward to using this tech when I am developing DOOM 5 exclusively on the Xbox 1080.

[Ali_The_Brit]

thats pretty damn amazing but just because its by microsoft you think it has anything to do with gaming or the xbox?

give me a break.

[TheAveragePs3User]

so stewpid! this is made only for facewrapping enthusiasts!!!11!!

[doctorb]

AMAZING

[Stickguy259]

Pretty darn cool.

[poeo]

damn, i really want to learn how to use that program and do that sort of stuff!

but hmm, it's kinda scary when you think about this technology being misused, soon we will not know what to believe when we look at news etc, any video can be a convincing fake... (yea i know a lot of news are bullsh*t already, but you know what i mean)

[Honolulu]

Cool,
But this has nothing to with games at all, this is just an automatic tracking technique as far as I can see that has too little precision for being of any use.

but there's always tons of stuff like this on siggraph every year, most of the techniques never finds any commercial use though.