Windows 8 comes with multiple input types. Windows 8 devices often have multi-touch screens that enable users to use multiple fingers simultaneously to produce different input interactions such as tapping, dragging, or pinching. The Windows Runtime has a number of different mechanisms for handling touch input, enabling us to create an immersive experience that we can explore with confidence. This Article covers the basics of using touch input (simulating the touch input via VC++ code) in Windows Developer Preview using C++ by Touch Injection API. For other (consumer preview and release preview) you may need to enable UI access permission( check references for this). Using this article will help all to create application that will help in simulating the Touch inputs like Tap, Drag, Rotate etc. , Consider an application that will convert users mouse event to touch event for those users who does not have Touch screen monitor. Also this will be of great use for Automating any touch based application on windows 8.
This Article assumes that Reader is updated with these basics requirements
Touch Interactions are a high-level way of interpreting touch input data into a set of common motions such as tapping, dragging, and pinching. Lets Understand some of the common interactions used in Windows Developer Preview which are as follow:
Interaction
Description
For windows 8 developer preview this will work as it is but for other windows 8 versions you may need to change UI access permission. Lets see how to simulate these Touch interaction via VC++ code using Touch Injection API.
By Tap we mean that it has simply touched any portion of screen. In this use case actor takes two action
To get above done you need to following 5 steps.
1] Create a Win32 application using visual studio 2011 developer preview. ( with No CLR support and without ATL support)
POINTER_TOUCH_INFO contact;
InitializeTouchInjection(1, TOUCH_FEEDBACK_DEFAULT); // Here number of contact point is declared as 1.
memset(&contact, 0, sizeof(POINTER_TOUCH_INFO));
contact.pointerInfo.pointerType = PT_TOUCH;
contact.pointerInfo.pointerId = 0; //contact 0
contact.pointerInfo.ptPixelLocation.y = 200; // Y co-ordinate of touch on screen
contact.pointerInfo.ptPixelLocation.x = 300; // X co-ordinate of touch on screen
contact.touchFlags = TOUCH_FLAG_NONE;
contact.touchMask = TOUCH_MASK_CONTACTAREA | TOUCH_MASK_ORIENTATION | TOUCH_MASK_PRESSURE;
contact.orientation = 90; // Orientation of 90 means touching perpendicular to screen.
contact.pressure = 32000; // defining contact area (I have taken area of 4 x 4 pixel)
contact.rcContact.top = contact.pointerInfo.ptPixelLocation.y - 2;
contact.rcContact.bottom = contact.pointerInfo.ptPixelLocation.y + 2;
contact.rcContact.left = contact.pointerInfo.ptPixelLocation.x - 2;
contact.rcContact.right = contact.pointerInfo.ptPixelLocation.x + 2;
contact.pointerInfo.pointerFlags = POINTER_FLAG_DOWN | POINTER_FLAG_INRANGE | POINTER_FLAG_INCONTACT;
InjectTouchInput(1, &contact); // Injecting the touch down on screen
contact.pointerInfo.pointerFlags = POINTER_FLAG_UP;
InjectTouchInput(1, &contact); // Injecting the touch Up from screen
By Hold it mean that Touch is continuously in contact to the screen. In this use case actor takes two action
To follow the above scenario repeat all 4 steps mentioned in TAP Simulation and then add the below code as last steps
//POINTER_FLAG_UPDATE when use with POINTER_FLAG_INRANGE and POINTER_FLAG_INCONTACT keeps the touch in drag mode with respect to last down Input. contact.pointerInfo.pointerFlags = POINTER_FLAG_UPDATE | POINTER_FLAG_INRANGE | POINTER_FLAG_INCONTACT;
for(int i=0;i<100000;i++){ //loops for approx. 1 second causing 1 second HOLD effect
InjectTouchInput(1, &contact);
}
contact.pointerInfo.pointerFlags = POINTER_FLAG_UP; // Moving the touch Up after Hold Complete
By Drag it mean that the user has touched down at any Screen co-ordinate and has moved his finger to another co-ordinate. In this use case the actor has three actions.
To follow above scenario repeat all 4 steps of TAP Simulation and then add the below code as last step.
//Setting the Pointer Flag to Drag
contact.pointerInfo.pointerFlags = POINTER_FLAG_UPDATE | POINTER_FLAG_INRANGE | POINTER_FLAG_INCONTACT;
for(int i=0;i<100;i++){
contact.pointerInfo.ptPixelLocation.x--; // updating the X Co-ordinate to x-100 pixels
// Lifts the touch input UP
By Pinch it means that we inject two contact point and drag both towards each other( Zoom-out) or drag both the contact point apart (Zoom-in). In this use case actor has to following three actions
To get the above scenario done follow the following steps.
//Taking two contact points and initializing the Touch API accordingly
POINTER_TOUCH_INFO contact[2];
InitializeTouchInjection(2, TOUCH_FEEDBACK_DEFAULT);
memset(&contact[0], 0, sizeof(POINTER_TOUCH_INFO));
memset(&contact[1], 0, sizeof(POINTER_TOUCH_INFO));
//Check Pointer Id is taken as 0 for contact 0
contact[0].pointerInfo.pointerType = PT_TOUCH;
contact[0].pointerInfo.pointerId = 0; //Id 0 for contact 0
contact[0].pointerInfo.ptPixelLocation.y = 200;
contact[0].pointerInfo.ptPixelLocation.x = 300;
//Defining Touch flag and touchmask for contact 0
contact[0].touchFlags = TOUCH_FLAG_NONE;
contact[0].touchMask = TOUCH_MASK_CONTACTAREA | TOUCH_MASK_ORIENTATION | TOUCH_MASK_PRESSURE;
contact[0].orientation = 90;
contact[0].pressure = 32000;
contact[0].rcContact.top = contact[0].pointerInfo.ptPixelLocation.y - 2;
contact[0].rcContact.bottom = contact[0].pointerInfo.ptPixelLocation.y + 2;
contact[0].rcContact.left = contact[0].pointerInfo.ptPixelLocation.x - 2;
contact[0].rcContact.right = contact[0].pointerInfo.ptPixelLocation.x + 2;
//Check Pointer Id is taken as 1 for contact 1
contact[1].pointerInfo.pointerType = PT_TOUCH;
contact[1].pointerInfo.pointerId = 1; //Id 0 for contact 1
contact[1].pointerInfo.ptPixelLocation.y = 200;
contact[1].pointerInfo.ptPixelLocation.x = 180;
//Defining Touch flag and touchmask for contact 1
contact[1].touchFlags = TOUCH_FLAG_NONE;
contact[1].touchMask = TOUCH_MASK_CONTACTAREA | TOUCH_MASK_ORIENTATION | TOUCH_MASK_PRESSURE;
contact[1].orientation = 90;
contact[1].pressure = 32000;
contact[1].rcContact.top = contact[1].pointerInfo.ptPixelLocation.y - 2;
contact[1].rcContact.bottom = contact[1].pointerInfo.ptPixelLocation.y + 2;
contact[1].rcContact.left = contact[1].pointerInfo.ptPixelLocation.x - 2;
contact[1].rcContact.right = contact[1].pointerInfo.ptPixelLocation.x + 2;
//Implementing two touch down for both array of contact point in one go
contact[0].pointerInfo.pointerFlags = POINTER_FLAG_DOWN | POINTER_FLAG_INRANGE | POINTER_FLAG_INCONTACT;
contact[1].pointerInfo.pointerFlags = POINTER_FLAG_DOWN | POINTER_FLAG_INRANGE | POINTER_FLAG_INCONTACT;
InjectTouchInput(2, contact);
//Setting both the contact point for Drag
contact[0].pointerInfo.pointerFlags = POINTER_FLAG_UPDATE | POINTER_FLAG_INRANGE | POINTER_FLAG_INCONTACT;
contact[1].pointerInfo.pointerFlags = POINTER_FLAG_UPDATE | POINTER_FLAG_INRANGE | POINTER_FLAG_INCONTACT;
for(int i=0; i<100;i++)
{
contact[0].pointerInfo.ptPixelLocation.x =300+i;//Zooming in by dragging apart both contact points w.r.t each other
contact[1].pointerInfo.ptPixelLocation.x =180-i;
contact[0].pointerInfo.pointerFlags = POINTER_FLAG_UP;
contact[1].pointerInfo.pointerFlags = POINTER_FLAG_UP;
To implement the above scenario repeat all the steps in shown in Pinch Simulation , but change the step 3 to the code shown below.
// implementing the rotation
if(i%3==0)
contact[0].pointerInfo.ptPixelLocation.x -= 3;
contact[1].pointerInfo.ptPixelLocation.x += 3;
contact[0].pointerInfo.ptPixelLocation.y = 200+i;
contact[1].pointerInfo.ptPixelLocation.y = 200-i;
To implement the above scenario repeat all the steps in shown in Pinch Simulation , but change the step 3 to the code shown below
contact[1].pointerInfo.ptPixelLocation.y = 200+i;
Richard Mueller edited Revision 20. Comment: Removed tag "MS"
Richard Mueller edited Revision 18. Comment: Replace RGB values with color names in HTML to restore colors
Sanjay Kumar Sinha edited Revision 17. Comment: Editing title
Richard Mueller edited Revision 16. Comment: Added TOC and tags
Richard Mueller edited Revision 15. Comment: Removed (en-US) from title, add tags
Sanjay Kumar Sinha edited Revision 12. Comment: containt management
Sanjay Kumar Sinha edited Revision 11. Comment: containt management
Sanjay Kumar Sinha edited Revision 10. Comment: containt management
Sanjay Kumar Sinha edited Revision 8. Comment: reverting to correct data
Nevin Janzen edited Revision 6. Comment: Tags Edit
Wonderful article, I have tried each snippet practically, and have found the successful outputs. Great Work Sanjay. Happy new year :)
Thanks nabarun.. happy new year :D
Simply awesome!!! More power to you!!!
Or you can use MultiTouchVista HID driver to emulate touch input and interact directly.
multitouchvista.codeplex.com
I am surprised that this works. According to the POINTER_INFO documentation ( msdn.microsoft.com/.../hh454907(v=vs.85).aspx ) in the "sourceDevice" section, it says "The POINTER_INFO source can never be NULL for any form of injected input." However, you are leaving the sourceDevice as NULL which is against the guidelines of the MSDN documentation. Is this a bug in the documentation or in your code?
Hi Adam,
Even in the start when I 1st tried to develop this code, I was having the same confusion, but when i worked out with Reference-1 of this article ( Touch Injection sample at code.msdn.microsoft.com/.../Touch-Injection-Sample-444d9bf7 ), I found the we can comfortably proceed with leaving sourceDevice to unassigned/null.
Hmmmm.. I guess the MSDN documents are according to the specs defined prior but the same check is not implemented in the TouchInjection API.... anyways :D
Thanks for the response Sanjay. I found that after you call InitializeTouchInjection, a touch device does show up that advertises the same number of touch points that I pass in to InitializeTouchInjection. Even if it doesn't matter much, maybe you could change your code to use GetPointerDevices to retrieve a handle to the device so you can inject into it. That would be more consistent with what MSDN advertises.
ya.. we can of course use GetPointerDevices and use that as input to source device.. but that will be only to please msdn doc ..:D , thanks for your suggestions ;)
Great!
I am working on writing UI automation tool for metro style app (record/playback style). This article would help in playback. Do you have any fo