Homography Puns

Hi good day all,

I am doing an augmented reality project currently. So I have got a 3x3 homography matrix using OpenCv findHomography function. Initially 3x3 homography matrix (Z axis column = 0) was for 2D projection, so I recover the Z axis column and it turns out to be a 3x4 homogrpahy matrix for 3d projection. Now I would like to turn the 3x4 homography matrix into OpenGl model view matrix. I did try with the transformation myself, but turns out its not working. I am not sure am I missing something for the transformation step or its completely wrong.

Code for transformation:

def transformation(img, homography_matrix, texture_file):
        # turn into 4x4 shape to match model view matrix in opengl
        view_matrix = np.vstack((homography_matrix,[0.0,0.0,0.0,1.0]))

        # transpose the matrix since opengl is in column major order
        view_matrix = np.transpose(view_matrix)

        # load texture
        init_object_texture(texture_file)

        # load the model view matrix
        glPushMatrix()
        glLoadMatrixd(view_matrix)

        # render the teapot object
        glutSolidTeapot(0.5)
        glPopMatrix()

Hi am working on a panorama stitching project with python, OpenCV and using orb to detect keypoints. I keep getting this weird behavior as I am trying to chain multiply the found homographies together. It's hard to describe but it is like a shrinking of the newly added image. You can somewhat see it as you look left to right in the image posted. Can anyone provide any insight? I am unsure about the problem causing this. I will gladly give more info if needed.

https://preview.redd.it/1gpxugif4w171.png?width=2507&format=png&auto=webp&s=eee7f865f2fab5090b624dfa8efdcba5f2d279fd

I have 4 points on an image and I know it is the projection of a rectangle. I also know the length of one side of this rectangle in the real world. Is this enough to automatically compute the length of the other side of the rect? I was thinking that maybe I could use optimization to find a homography matrix that satisfies the constrains (90 deg angles after transform and length of the one side I know)

Can anyone give me some intuition where and when to use homography and fundamental matrix?

background: I'm using feature detector and matcher to calculate homography between two consective frames in a video, so that I can put a logo on the first frame and track the positions on next frames. The result is basically ok, but the logo tend to jump a little bit up and down due to noise in the estimated homography, I'm thinking of using Kalman filter to smooth out the noise, but unlike quaternion, the homography matrix cannot be linearly interpolated, the result is rather bad。

I'm asking if there's a suitable representation of homography matrix that can be used in kalman filter?

I'm using a RGB-D camera (Intel Realsense D345) to implement a table top projected augmented reality system. Using chessboard calibration I obtain a transformation matrix which I use to transform each incoming frame using warpPerspective from openCV. It works really well for the color frames. The problem is, am I allowed to do this for depth images as well ? Considering depth images are 3D geometrical data. What's the right way to apply a transformation matrix to depth images?

Suppose I want to find a homography between a radar ground plane (lateral_position, distance, 1) and the image plane (x, y, 1) such as in this paper. How do you find the scale ambiguity? No one seems to go into that, am I missing something?

https://preview.redd.it/saelb40s0cg51.png?width=960&format=png&auto=webp&s=a7c98c28f8813b3e1b72296b420456fc8e98a56f

Today we are releasing a new version for Kornia that includes different functionalities to work with 3D augmentations and volumetric data, local features matching, homographies and epipolar geometry.

In short, we list the following new features:

• Support to PyTorch v1.6.0.
• Local descriptors matching, homography and epipolar geometry API.
• 3D augmentations and low level API to work with volumetric data.

Local features matching

We include an kornia.feature.matching API to perform local descriptors matching such classical and derived version of the nearest neighbor (NN).

https://preview.redd.it/cu9cg0fp0cg51.png?width=594&format=png&auto=webp&s=dbfc6154698435a9447865abfe171d144477dfba

https://preview.redd.it/x3c5c5q71cg51.png?width=658&format=png&auto=webp&s=cbcd9ae539871f56dea2e489685950fb0ce3ac96

Homography and epipolar geometry

We also introduce kornia.geometry.homography including different functionalities to work with homographies and differentiable estimators based on the DLT formulation and the iteratively-reweighted least squares (IRWLS).

https://preview.redd.it/qc8sfcf51cg51.png?width=1200&format=png&auto=webp&s=446cc11c36e7ac29e65c6f118c978c6d7fe1ecc2

https://preview.redd.it/236bv8881cg51.png?width=657&format=png&auto=webp&s=93ccb4319b0d52e605ba4b16c8ddf73dfd7da1d9

In addition, we have ported some of the existing algorithms from opencv.sfm to PyTorch under kornia.geometry.epipolar that includes different functionalities to work with Fundamental, Essential or Projection matrices, and Triangulation methods useful for Structure from Motion problems.

3D augmentations and volumetric

We expand the kornia.augmentaion with a series of operators to perform 3D augmentations for volumetric data.

https://i.redd.it/48hqrpd91cg51.gif

In this release, we include the following first set of geometric 3D augmentations methods:

• RandomDepthicalFlip3D (along depth axis)
• RandomVerticalFlip3D

Let's suppose i have a Homography matrix (H) computed between images A and B.
If I scale image A from its original resolution. Is it still possible for me to find H' (Homography between scaled A and original B) with only having H, Original resolution of both images, and scaled resolution of image A?

I am new to this topic, any help would be appreciated.

So I am building a AR sandbox which can run wind simulation. I got intrinsic matrix of Kinect and extrinsic matrix from using the calibration method as shown in video.

https://www.youtube.com/watch?v=EW2PtRsQQr0

I am using the calibration method from Oliver Kreylos's sandbox from UC davis to get these matrices

I did go through the pinhole camera model but am just not sure how to go about calculating the homography matrix from the two matrices mentioned.

Any help or even pointing me in the right direction would be much appreciated ! :)

import cv2
import numpy as np 
import pygame

pygame.init() # initializing pygame object 
infos = pygame.display.Info() #Display info of current machine's screen
screen_size = (infos.current_w, infos.current_h)



''' Intrinsic Parameters taken as constants. This is reasonable assumption since most Kinect cameras are around the same calibration parameters based on their factory settings '''

'''fx = 594.21
fy = 591.04
a = -0.0030711
b = 3.3309495
cx = 339.5
cy = 242.7
int_mat = np.array([[1/fx, 0, 0, -cx/fx],
                    [0, -1/fy, 0, cy/fy],
                    [0,   0, 0,    -1],
                    [0,   0, a,     b]]) '''
# Intrinsic Camera matrix plus radial/tangential distortion functions
int_mat = np.matrix ([[ 1.68290672e-03,  0.00000000e+00,  0.00000000e+00, -5.71346830e-01],
                      [ 0.00000000e+00, -1.69193286e-03,  0.00000000e+00,4.10632106e-01],
                      [ 0.00000000e+00,  0.00000000e+00,  0.00000000e+00,-1.00000000e+00],
                      [ 0.00000000e+00,  0.00000000e+00, -3.07110000e-03,3.33094950e+00]] )
#Extrinsic Matrix plus radial/tangential distortion functions 
ext_mat = np.matrix([[ 1544.99   ,   39.1753 ,    -506.749 ,       60258],
                     [-134.243 ,     2150.39 ,     253.107 ,     89226.2],
                     [-0.074654 ,  -0.0998495 ,   -0.992198  ,    103.256]])

h_mat =np.array([[ 3.09221242e+00,  1.76363777e-01,  2.45422191e-03,  1.44350001e+01],
       [-2.74937775e-01,  5.69983305e+00,  1.65133063e+00,  1.29103457e+02],
       [-1.20148608e-01, -1.60698301e-01, -1.59684918e+00, -8.27651000e+01],
       [-7.46540183e-02, -9.98494623e-02, -9.92197996e-01,  1.03256412e+02]])

warp = cv2.warpPerspective(frame, h_mat, screen_size) # frame is a 

I am currently using homography to map live soccer camera feed frame to a 2d plane. The aim is to detect the locate the player on that 2D plane. I have successfully mapped the area however i am wondering if there is any way i can find location of a single pixel on the wrapped image.

Objective: I want to find the new coordinates of point F(x,y) from the feed on the transformed Image.

Current Approach: Currently, after finding the homography matrix of the current frame i make transPlane (a black equally sized frame with RGB dots on the centroids of player bounding boxes). Which is then passed to a cv2 function

--cv2.warpPerspective(transPlane, h, dst_size, borderMode=cv.BORDER_CONSTANT, borderValue=background_color)

- [TransPlane](https://i.imgur.com/L7NZGhR.png) : (a black equally sized frame with RGB dots on the centroids of player bounding boxes)

-h: (3 x 3) Homography Matrix

-dstSize: (115,74)

Return from that image is a [wraped Image](https://i.imgur.com/GEY3Pan.jpg) with size(115,74). After that i apply connected components algorithm to identify the location of the centroids on the wraped image. I want to directly translate the points on the plane.

- [TransPlane] (https://i.imgur.com/L7NZGhR.png)

- [wraped Image](https://i.imgur.com/GEY3Pan.jpg)

I am trying to compute the Homography without using the Opencv library.

I am doing as mentioned in this link:

https://math.stackexchange.com/questions/494238/how-to-compute-homography-matrix-h-from-corresponding-points-2d-2d-planar-homog

Also, went through the link below :

https://alyssaq.github.io/2015/singular-value-decomposition-visualisation/

After getting a matrix from the above link, say matrix A

I am calculating the SVD of A using

S,U,V = np.linalg.svd(A, full_matrices=True)

and the homography matrix H then is the last column of conjugate of V transpose.

But the answer given by cv2.findHomography and my above method dont seem to match.

Can someone help me with this?

I'm not a stranger to programming, but usually I work more with audio rather than video and images. I've been working on a little personal project that involves augmented reality. I messed around with different marker tracking methods and found that working with Aruco markers (which are included in Opencv) works the best so far.

TL,DR: what are generally the different techniques to put a 3D model into a scene? And most importantly which ones are applicable when I only have a single square (Aruco) marker?

Hi!

I am trying to accomplish a precise spherical stitching using OpenCV, with the final goal to create a full 360x180 panorama. But let's start simple with 2 images:

• I have SURF matches : https://i.imgur.com/lST3CTL.png

• From which using classic RANSAC method, I get an homography H. I am veryfing the quality of this homography by using a simple planar projection: https://i.imgur.com/rz8UKwv.png

• Then, enters the camera intrinsic parameters matrix K. Knowing the FoV of my camera and the width/height of my images (they're all the same), it's easy to get K: I compute the focal using image_width * 0.5 / tan(FoV * DEG2RAD * .5)

• I compute a rotation matrix R' = K.inv() * H * K, then apply SVD onto R' to get R = U * VT. Using this rotation matrix, and under the assumption that the rotation of my first image is the Identity matrix, I can apply an equirectangular projection on both my images and stitch them together (no seamless blending for now, I want to see if my images do overlap perfectly): https://i.imgur.com/0B6aACP.png

As you can see, the result is pretty good! Unfortunately I am having trouble understanding why the same process doesn't work for all my images. For instance:

• Here are the SURF matches on another pair: https://i.imgur.com/eRYMg34.png

• And the resulting planar projection: https://i.imgur.com/9inS3xX.png

• Still pretty good right ? Well, with the same K matrix, the equirectangular projection is... not so good: https://i.imgur.com/W2GSgEb.png

It seems like I need to change my focal (and K) to get the result I want. But why? The pictures have been taken with the same camera, with pure rotation, there should not be a different focal length. Also, if I want to get at least a full cylinder, I will have issues in my final blend if I need a different focal length between pairs: the last image will never connect with my first image correctly.

Any help on the matter?

Thanks a lot ! :)

Edit: By the way, yes I have seen stitching_detailed in opencv, but this is not very practical for a large amoun

I posted earlier regarding translating location of a player using the homography matrix of the camera angle and u/aNormalChinese proposed that I should use cv2.perspective transform

I used the following method to input the coordinates on using the homography matrix. The proposed solution was to use cv2.perspectiveTransform( ptsOrignal, HomographyMatrix). This function returns a list of a list of the number of points, each of size 512.

cv2.perspectiveTransform

Moreover, earlier regarding the translating location of a player using the homography matrix of the camera angle and dimensional (x,y,1) and using the third dimension to normalize the new coordinates will give me the translated coordinates. But this gives me one negative and a positive incorrect coordinate

I want the updated coordinates of the player after transformation. I know this is something I should post on StackOverflow but given there are some people here who might have worked with this and they helped me earlier with this I am hoping that I 'll get my issue resolved 🤞

Hi guys!$

I am trying to calculate an homography but I can't seem to get the right results:

Here is my code:

width, height = 812, 390

origin = [[814, 526], [-540, 297], [506, 207], [1074, 222]]
dest = [[0, 0], [0, height], [width, height], [width, 0]]



def getPerspectiveTransformMatrix(p1, p2):
    A = []
    for i in range(0, len(p1)):
        x, y = p1[i][0], p1[i][1]
        u, v = p2[i][0], p2[i][1]
        A.append([x, y, 1, 0, 0, 0, -u * x, -u * y, -u])
        A.append([0, 0, 0, x, y, 1, -v * x, -v * y, -v])
    A = np.asarray(A)
    U, S, Vh = np.linalg.svd(A)
    L = Vh[-1, :] / Vh[-1, -1]
    H = L.reshape(3, 3)
    return H

H = getPerspectiveTransformMatrix(origin, dest)

print("origin: -540; 297, target 0, 390")
print(np.matmul(H, np.array([[-540], [297], [1]])))
print("origin: 506; 207 target 812 390")
print(np.matmul(H, np.array([[506], [207], [1]])))
print("origin: 1074; 222 target 812 0")
print(np.matmul(H, np.array([[1074], [222], [1]])))

I was thinking that if everything else is right, I should get dest = H*origin, but I seem to be missing something since the output of this code is:

origin: -540; 297, target 0, 390
[[-1.03027560e-08]
 [-3.54223022e+02]
 [-9.08264158e-01]]
origin: 506; 207 target 812 390
[[-276.39190269]
 [-132.74980549]
 [  -0.34038412]]
origin: 1074; 222 target 812 0
[[-3.60337203e+02]
 [ 5.11590770e-13]
 [-4.43765029e-01]]

When using OpenCv to compute H, I get the same results which suggests my problem is in the understanding of what my output should be ?

Any help would be great ! :)

I'll edit the post if I get anything more promising.

I'm very new to OpenCV and still trying to figure out where I can find resources, so I appreciate the patience! I'm wondering if OpenCV has a way to find the homography of a QR Code or some other tracker, given a video. I want to be able to put a tracker on a table and return the homography of the tracker relative to the perspective of the camera. Things that aren't OpenCV would work also--I'd actually prefer something that could do this in Matlab.

Thanks!!

Assume I have a stereo image pair with known extrinsics and intrinsics. I can use OpenCV's [stereoRectify()](https://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html#void%20stereoRectify(InputArray%20cameraMatrix1,%20InputArray%20distCoeffs1,%20InputArray%20cameraMatrix2,%20InputArray%20distCoeffs2,%20Size%20imageSize,%20InputArray%20R,%20InputArray%20T,%20OutputArray%20R1,%20OutputArray%20R2,%20OutputArray%20P1,%20OutputArray%20P2,%20OutputArray%20Q,%20int%20flags,%20double%20alpha,%20Size%20newImageSize,%20Rect*%20validPixROI1,%20Rect*%20validPixROI2)) to get rotation matrices that transform the camera coordinate systems into rectified ones. Also the function returns the new projection matrix in the rectified system.

Two questions: How can I calculate the corresponding rectifying (image-) homographies H, so that (x_rect * w, y_rect * w , w) = H * (x, y, 1)? And how can I calculate the fundamental matrix after rectification?

Sorry if this is a trivial question but I could not find a suitable answer elsewhere. I recently learned about homography and fundamental matrices in a CV class. I understand how these matrices are calculated and how the the formulae are derived. I also understand that both homography and fundamental matrices have 8 degrees of freedom. But I'm unable to understand why we need 4 point correspondences for finding the homography while 8 points for calculating the fundamental matrix. Shouldn't the number of correspondences needed be same intuitively? I'm definitely missing something and would be grateful if someone can help me understand this problem or correct my understanding. Thanks!

https://preview.redd.it/coyyuk8vjcg51.png?width=960&format=png&auto=webp&s=4a4f9cd02434de96e0e130ad8d7b7b3bc48b2e7a

We are releasing a new version for Kornia that includes different functionalities to work with 3D augmentations and volumetric data, local features matching, homographies and epipolar geometry.

In short, we list the following new features:

• Support to PyTorch v1.6.0.
• Local descriptors matching, homography and epipolar geometry API.
• 3D augmentations and low level API to work with volumetric data.

Local features matching

We include an kornia.feature.matching API to perform local descriptors matching such classical and derived version of the nearest neighbor (NN).

https://preview.redd.it/ma7ghuawjcg51.png?width=594&format=png&auto=webp&s=a1a8cd7c473fbf6883c1104781819ccbffe72acf

https://preview.redd.it/mjf5wswwjcg51.png?width=658&format=png&auto=webp&s=966b8827b5844c964e8553fca15abc64eced2331

Homography and epipolar geometry

We also introduce kornia.geometry.homography including different functionalities to work with homographies and differentiable estimators based on the DLT formulation and the iteratively-reweighted least squares (IRWLS).

https://preview.redd.it/18or79xzjcg51.png?width=1200&format=png&auto=webp&s=47024a11c1b6258cd22f3fb384615ee5c63d0d37

https://preview.redd.it/jvicn10zjcg51.png?width=657&format=png&auto=webp&s=941d865cab78ce12e12f3f1f7afca9ad5b0bd961

In addition, we have ported some of the existing algorithms from opencv.sfm to PyTorch under kornia.geometry.epipolar that includes different functionalities to work with Fundamental, Essential or Projection matrices, and Triangulation methods useful for Structure from Motion problems.

3D augmentations and volumetric

We expand the kornia.augmentaion with a series of operators to perform 3D augmentations for volumetric data.

https://i.redd.it/1pe7y6u0kcg51.gif

In this release, we include the following first set of geometric 3D augmentations methods:

• RandomDepthicalFlip3D (along depth axis)
• RandomVerticalFlip3D (along height axis)
• RandomHoriz

https://preview.redd.it/ok4a1cui3cg51.png?width=960&format=png&auto=webp&s=448f66b17c61d5fe4dbf026e12ccf42f6dc5a64d

Today are releasing a new version for Kornia that includes different functionalities to work with 3D augmentations and volumetric data, local features matching, homographies and epipolar geometry.

In short, we list the following new features:

• Support to PyTorch v1.6.0.
• Local descriptors matching, homography and epipolar geometry API.
• 3D augmentations and low level API to work with volumetric data.

Local features matching

We include an kornia.feature.matching API to perform local descriptors matching such classical and derived version of the nearest neighbor (NN).

https://preview.redd.it/8muh96dj3cg51.png?width=594&format=png&auto=webp&s=93e00208afb29941e2f8b51aa0c02cbc015e2418

https://preview.redd.it/8zosovmj3cg51.png?width=658&format=png&auto=webp&s=a0c74c179652743a24c2b6c3ba588c4caa038592

Homography and epipolar geometry

We also introduce kornia.geometry.homography including different functionalities to work with homographies and differentiable estimators based on the DLT formulation and the iteratively-reweighted least squares (IRWLS).

https://preview.redd.it/0exbmwok3cg51.png?width=1200&format=png&auto=webp&s=10d9b8b2f4d3b9a2e2bc340ece2722521f3936f3

https://preview.redd.it/wx1of1yk3cg51.png?width=657&format=png&auto=webp&s=51f7d676b6a735fb9cc628fac5ead484087e73d5

In addition, we have ported some of the existing algorithms from opencv.sfm to PyTorch under kornia.geometry.epipolar that includes different functionalities to work with Fundamental, Essential or Projection matrices, and Triangulation methods useful for Structure from Motion problems.

3D augmentations and volumetric

We expand the kornia.augmentaion with a series of operators to perform 3D augmentations for volumetric data.

https://i.redd.it/sy3lviwl3cg51.gif

In this release, we include the following first set of geometric 3D augmentations methods:

• RandomDepthicalFlip3D (along depth axis)
• RandomVerticalFlip3D (alo