How to create a T-shirt or book displacement map in practice?

Question

So I learned that the feature I am interested in is called a "displacement map". This makes it so you can take a blank t-shirt (with all it's curves and subtle textures), and apply an image to it so it looks like the t-shirt actually has that image on it.

The part I'm wondering about is how do you make this a dynamic process? How do you, say, take a book cover at an angle with specific lighting, or a model wearing a t-shirt in specific lighting at a specific angle, and apply the graphic? How could I go about creating such a "template" image of a blank book or model with t-shirt, such that I can upload an image and it gets nicely applied to the t-shirt and all I have to specify is the width/height/x/y of the image? How do you make it so the image gets rotated/skewed properly? What sorts of data models am I dealing with here?

I am a software engineer but I've never had to deal with such a problem before and it seems like I would have to create something like this... Somehow create a 3D model out of the image I take of a model wearing a T-shirt, turn this 3D model into a displacement map or UV texture or (throw some more graphics jargon at it which I don't fully understand) and then the image will become a texture on a 3D image. This means I have to do a lot of work and sort of sculpt a 3D model out of an image, because I don't see how it could be done automatically....

That's as much as I can imagine. Wondering if one could gently guide me through the process, not in too much detail, but in some detail. I would like to create something like this or this.

Answer 1

This is just a speculation. I don't know how these algorithms work. But I can imagine a possible implementation based on the video and other examples of how this effect works. Take note that this kind of algorithm is just an approximation, or a fake that looks kind of believable. If you wanted a photo-realistic approach, the approach with 3D model, texture mapping and rendering would be better approach.

First, you need a "displacement map", a grayscale image that will define how the displacement work. In the video, this displacement is generated from the image itself, but it can also be hand-drawn or created using a complex filter from the image itself. The best way to think of this image is as a height-map. The darker the value, the "farther out" from the image the pixel would be.

Second step would be to generate a "surface tangent map". I find this kind of similar to normal map, but instead of the vector pointing away from the surface, the vector at each pixel will point in direction of the slope of the height map. The way to calculate this would be to look at a pixel, and according to it's neighbor pixels, calculate "direction" of the slope as defined by grayscale "height".

Third step is simple, simply apply the "surface tangent map" on the image, displacing the original pixels by the amount of the vector in the tangent map.

Tried to make an explanation image for 1D scenario. But it shouldn't be difficult for extend to 2D. You just need to play around with the vector math.