Texture Paper Reading[Let There Be Color!]
 Let There Be Color!  LargeScale Texturing of 3D Reconstructions
 Paper.pdf

In this paper we therefore present the first unified texturing approach that handles large, realistic datasets reconstructed from images with a structurefrommotion plus multiview stereo pipeline.

millions of triangles within less than two hours.
 Related
 view selection
 blend multiple views per face[5,13]
 one view per face[9,10,15,23]
 one view per texel[19]
 one view per face but blend close to texture patch borders[2,6]
 single view per face based on a pairwise Markov random field[15]
 smoothness term models
 view blend
 weight blending[5]
 masks indicating
 angle to weight

suggest the use of vertex colors in combination with mesh subdivision
 blending in frequency space[2,6]
 weight blending[5]
 blend pixels based on angle and proximity to the model[13]
 viewdependent texturing— Lumigraph[4]
 Color Adjustment
 local
seam's mean
[23] heat diffusion
 global
 globally optimal luminance correction terms[15]

After adjustment luminance differences at seams should be small and the derivative of adjustments within a texture patch should be small

 globally optimal luminance correction terms[15]
 local
 view selection
 Assumptions and Base Method
 Pairwise Markov random field energy formulation[15]

$$ \begin{aligned} E(l)&=\sum_{F_i\in Faces}E_{data}(F_i, l_i)+\sum_{F_i\in Faces}E_{smooth}(F_i, F_j, l_i, l_j)&(1) \end{aligned} $$
 $l$: labeling of views
 $F$: mesh face
 $E_{data}$:
good
views for texturing a face
base method: the angle between viewing direction and face normal.

project a face into a view and use the projection’s size as data term[2]

Lumigraph’s view blending weights face normal[4] 💩

the gradient magnitude of the image integrated over the face’s projection[9]
 to solve
closeup the faces closest to the camera
thatnot be in focus
andlead to a blurry texture
.  $x,y \pm64pixel$
 to solve

 $E_{smooth}$: minimizes seam (i.e., edges between faces textured with differentimages) visibility.
 $E(l)$minimized with graph cuts and alpha expansion[3]
 This Paper [Compute seam color]

We found, that computation of the seam error integrals is a computational bottleneck and cannot be precomputed due to the prohibitively large number of combinations. Furthermore, it favors distant or lowresolution views since a blurry texture produces smaller seam errors, an issue that does not occur in their datasets.
 Add a
additive correction
$g$ 
$$ \begin{aligned} &\underset{\mathbf{g}}{\argmin}\sum_v \big(f_{v_{left}} + g_{v_{left}}  (f_{v_{right}}+g_{v_{right}})\big)^2 + \frac{1}{\lambda}\sum_{v_i,v_j}(g_{v_i}g_{v_j})^2&(2) \end{aligned} $$
 $v$: vexel
 $v_{left}$: vexel at left of seam
 $v_{right}$: vexel at left of seam
 $v_i,v_j$: vexel at same seam and
adjacent
 $f_{v*}$: color of vexel
 $g$: additive correction

After fnding optimal gv for all vertices the corrections for each texel are interpolated from the $g_v$ of its surrounding vertices using barycentric coordinates.

 LargeScale Texturing Approach
 Preprocessing
 check
face visibility
byColNet
[11]  precompute the data terms for Equation $(1)$
 check
 View Selection
 graph cuts and alpha expansion[3]
 replace[9] the
Data term
to $E_{data} = \int_{\phi(F_i,l_i)}\Vert\nabla(I_{l_i}(p))\Vert_2dp$💩
 $F_i$: face
 $\phi(F_i,l_i)$: $F_i$’s projection
 $I_{l_i}$: Sobel operator
 $\Vert\nabla(I_{l_i})\Vert_2$: gradient magnitude
 大概意思就是，面投影到图片，计算投影区的
Sobel operator
, 如果投影区太小没有像素，就gradient magnitude at the projection's centroid and multiply it with the projection area.
, 最后把它们加一块
 $E_{data} = \int_{\phi(F_i,l_i)}\Vert\nabla(I_{l_i}(p))\Vert_2dp$💩
 PhotoConsistency Check [检查遮挡和行人]

introduce an additional step to ensure photoconsistency of the texture.
 assuming

the majority of views see the correct color.

A minority may see wrong colors

 reject inconsistent views
 mean[19]
 median[13]
 meanshift algorithm[this]
 Compute the face projection’s mean color $c_i$ for each view $i$ in which the face is visible.
 Declare all views seeing the face as inliers.
 Compute mean $\mu$ and covariance matrix $\Sigma$ of all inliers’ mean color $c_i$.
 Evaluate a multivariate Gaussian function $exp(\frac{1}{2}(c_i\mu)^T\Sigma^{1}(c_i\mu))$ each view in which the face is visible.
 Clear the inlier list and insert all views whose function value is above a threshold $6E3$ as default
 Repeat 3.5. for
10
iterations or until all entries of $\Sigma$ drop below $10E5$, the inversion of $\Sigma$ becomes unstable, or the number of inliers drops below 4

 replace the
Smoothness Term
 based on the
Potts
model  $E_{smooth} = [l_i\neq l_j]$💩
 $[\cdot]$: is the
Iverson bracket
, 满足条件就是1，不满足就是0.
 $[\cdot]$: is the
 based on the
 Color Adjustment
 local adjustment with Poisson editing[16]
 global adjustment[15] problem

the vertex v’s projection into the two images adjacent to the seam. If there are even small registration errors (which there always are), both projections do not correspond to exactly the same spot on the real object. Also, if both images have a different scale the looked up pixels span a different footprint in 3D. This may be irrelevant in controlled lab datasets, but in realistic multiview stereo datasets the lookups from effectively different points or footprints mislead the global adjustment and produce artifacts.
 Color Lookup Support Region
 $v_1$点的颜色等于两条边的颜色*距离权重
 Equation 2 matrix form

$$ \mathbf{\Vert Agf\Vert_2^2+\Vert\Gamma g\Vert_2^2=g^T(A^TA+\Gamma^T\Gamma)gf^TAg+f^Tf} $$
 $\mathbf{f}$: vector, stacked $f_{v_{left}}  f_{v_{right}}$
 $\mathbf{A}$: sparse matrix with $\pm 1$
 $\mathbf{\Gamma}$: sparse matrix with $\pm 1$
 minimize it with
conjugate gradient
when $\frac{\Vert r\Vert_2}{\Vert\mathbf{A^Tf}\Vert_2}<10^{5}$ $r$: the residual
 local adjustment with Poisson editing[16]
 Poisson Editing
 Have a look at UCAS.IO/PossionImageEdit
 seams adjustment

perform local Poisson image editing[9, 16]

Poisson editing of a patch to a 20 pixel wide border strip[this]


 Preprocessing
 Evaluation
 Data Term and PhotoConsistency Check
 Smoothness Term
 Color Adjustment
 Data Term and PhotoConsistency Check