How to transform basic OpenCV shapes using pose / projection matrix?

0 votes
asked Mar 15, 2016 by neoknip (410 points)


I am trying to draw basic shapes of OpenCV shapes (triangle, circle) etc. using the sample application.

I am able to draw the shapes when objects are found on the camera view (rgba).

But I am not sure how to draw the line over the target as it moves around the camera view. 

If i need to move or rotate the rectangle and draw it based on the detected object, how can i achieve it?

Can I use the pose() of the tracker to do this? If Yes, can you tell me how?

Can I draw 3D objects as well using OpenCV?

Code snippet

public Mat onCameraFrame(CvCameraViewFrame inputFrame) 
    mRgba = inputFrame.rgba();
    // track every new frame provided by OpenCV's camera...
        if (mObjectFound) {
            Core.rectangle(mRgba, new Point(50.0, 100.0), new Point(100.0, 50.0), new Scalar(255, 0, 0));
   return mRgba;


1 Answer

0 votes
answered Mar 15, 2016 by Alex (6,680 points)
selected Mar 18, 2016 by neoknip
Best answer


in order to draw any content registered with the real objcet that you want to track, you need to get the pose from the tracker. The pose is basically a matrix representing the 3D transformation of the camera with respect to the reference system of the tracked object.


To use the pose you have three possibilities:


  1. you use the Rendering module provided with the SDK, in this case you do not need to know too much about 3D transforms and geometry, just add models to the scene and they will be trasformed accrodingly to the actual pose (with this approach you can load models from files but you cannot just "draw" lines, shapes, ...)
  2. you implements a rendering engine using the graphics APIs you prefer, for instance OpenGL ES 1, 2 ,3 or higher level APIs provided by your operating system, in this case you must have some experience with Real-time 3D Graphics programming (with this approach you have full control on the content so you can draw whatever you want)
  3. you use the Unity 3D plugin through which you can create the augmented content visually, in this case you should have a basic knowledge of Unity 3D of course


You could use OpenCV to draw lines and shapes on top of the camera frame in a manner that simulates a 3D geometry, but you'd still need to start getting the tracker's pose and then project 3D points on the camera frame which is a 2D entity...I'd strongly recommend one of the three approaches listed above.


Hope this helps.


commented Mar 17, 2016 by neoknip (410 points)
Thanks for the detailed response.
1) For Rendering module provided with the SDK, Can you provide a small example of placing a model at a specific position in reference to the tracked object.
2) For OpenGL, Is it possible to integrate it with the OpenCV's MAT object? Any samples would be helpful as well

I understand I can use openscenegraph models (.osg) using the Rendering APIs of the SDK.
Thanks for the help Alex.
Welcome to Developer Portal Q&A, where you can ask questions and receive answers from other members of the community.