So we have our tablets , our phones , our ageing consoles , and uber pcs recently made irrelevant by cloud services like onlive and Gaikai and Facebook apps and Angry Bird Farming sims ,but where do we go from here ? Well the personal viewing device is set to make another comeback ...
HMDs are THE thing right now , how do we know ? Because anything that is the next big thing in the world of tech is out of stock or still rumoured to be vaporware and or receiving lukewarm reviews from confused tech reviewers . Seriously when Sony announced their HMZ T1 concept at CES earlier this year even they thought they had a niche product , but fans , groomed on tales of Panther Moderns , girls with killer nails called Molly and Rastafari in space stations , fans who also may have a cat or hamster named Wintermute...well those fans have just booted the HMD revolution . For Sony this could be a potential Walkman moment . Other pioneers like siliconmicrodisplay and Carl Zeiss Optics are also enthralling and exciting would be cyberpunks bringing transparent 1080p and lightweight 720p portable headsets later this year and early next year.
But this is not a tech review, for thegameveda has yet to receive a Sony HMZ T1 ... so till then I will ponder how one chooses ones HMD and it is astonishingly beautifully simple : You do it based on your favourite Anime/Scifi allegiance:
1. Sir or Madam opts for the Sony HMZ T1 :the choice of the Star Trek , Ghost in the Shell , and DaftPunk/Tron Fan
The Sony offering gets you a :
HMZ-T1 Head Mounted Display
A personal cinematic home entertainment experience
Dual OLED HD screens, virtual 5.1 surround sound, TruBlack Display
Watch movies and games in High Definition 2D & 3D
Bright, clear, dual OLED screens
High contrast and clear images with TruBlack
Virtual 5.1 surround sound
First and foremost all fans of this fellow need this :
and taking a look at the unit itself , not only does it give you that Laforge vibe , it actually looks like a Start Trek next gen ship....
which is what clinched it for me :
looks just like my STO ship....
only now I get to wear it !
The ultra cool amongst you will immediately be thinking of the delectable Major Motoko Kusanagi:
thanks to the HMZ T1 you can go back and play Ghost In the Shell on the PlayStation , just like Major Kusanagi ...just don`t let anyone see you do it!
and then of course we come to the Daft Punk /Tron fans you need a Sony HMZ-T1 if your walls are adorned with anything resembling the following :
Now Sony`s is not the only HMD game in cyberspace
2.The most interesting challenge to Sony`s personal viewer , is siliconmicrodisplay`s ST1080 with true 1080p, see through and it`s greatest difference to the Sony HMZ T1 is it`s mobility , it`s weight being just 180 grams , and it`s power coming from USB.
there is a caveat to the 1080p claim , as no console can actually play 3D games at 1080p you are in fact still locked to 720p for 3D and 2D console games . Also how the LCoS 10% see through display stacks up to the Sony OLEDs is going to be very interesting. But for personal wearable computing fans this is going to be a boon.
In fact all these devices really shine with the power of a PC connected to them .
The ST1080 of course will be the choice of fans of this fellow
3.Last but not least we have the Carl Zeiss Cinemizer OLED 720p 3D Glasses also announced at the same time as the Sony HMZ T1 , but with a release date slipping to next year
here is one lucky cyberpunk testing out a prototype
Like the ST1080 the Cinemizer 720P is portable and can work off battery.
this will be the HMD choice for Doc Brown fans everyhwere :
All 3 of these products have been promised before Christmas 2011 , Sony has actually delivered , we will find out how the siliconmicrodisplay launch goes on Dec 2nd , and there is still nothing on the Carl Zeiss site about the cinemizer .
At present the HMZ T1s are the hottest tech globally this side of Christmas , of course you could argue they are nearer prototypes than the usual slick finished article the Playstation and iPhone generation are used to . But early adopters of these devices know this , and it is intriguing why Sony`s HMD is so bulky , and expensive compared to the promised price of the the ST1080 and the Cinemizer OLEDs .
Even if the ST1080 and Cinemizer OLEDS launch this side of Christmas they will not be able to do so internationally.
At this point I would have to say the fact that Sony are behind the HMZ T1 , and actually have had the product out for over a month , and incidentally sold out, swings it for now . The ST1080 sounds great , but 10% see through , and no option for total immersion, plus the fact that a lot of content is going to be limited to 720p any way , and most importantly this is a first gen product from a new company ,already making promises of a 2nd gen product whilst yet to release a first gen one , to most people makes the Sony option the safe bet at present. Likewise no one has actually seen or used the Carl Zeiss cinemizer outside trade shows , the above video and one with some nice German girls demoing it , or should I say modelling it .
I am weary of new start ups , and long term announcements of hardware that does not show up ,so I wait for the ST1080 December 2nd outing with a little trepidation. And you can google the Cinemizer OLEDs all you like at present and just find vaporware previews and hopes of fans ...
So it is great to see a company as large as Sony take a chance on something like this,bring it , confuse the old tech reviewers and manage to channel your inner cyberpunk , this is not going to be for everyone , but for those people that have already caused the HMZ T1 to sell out , it really is the gadget of the year .
Head tracking is another way the human / VR interface is becoming reality . Racing games can benefit greatly when you get to look around your virtual car . Today I used pppjoy mouse emulation to give me mouse control in F1 2012 mapping the x axes to look left and right. And as on the basic level the Zeiss headtracker emulates mouse control ,we have headtracking enabled in F1 2012. So how does the addition of headtracking change playing F1 2012?
Driving using headtracking turns this console port into a driving sim , right now I am using mouse emulation to get headtracking , what we need is Codemasters to introduce full axes headtracking to further enhance this series and take advantage of devices like the Cinemizer OLED headtracker . More to follow
Two of the finest games ever created , played with Carl Zeiss headtracking and even in Tridef 3D using the Carl Zeiss Cinemizer OLED headtracker.
System Shock is one of the most revered games of all time , achieving more in a a couple of floppy disks than today's 30 gig installs..
The Zeiss headtracker is compatible with thousands of games all without a single driver install, including classics like the System Shock series... so let's dive in and check out the original, time to navigate cyberspace wearing a display and using headtracking!
and now the finest game you can buy at the moment : System Shock 2 played in Tridef 3D using headtracking with the Cinemizer OLED hmd + headtracker incidentally this game rockets along at 2560 by 1600 using the Intel HD3000!
In a week where we are about to face PS4 , Crysis 3 and MetalGearRising Revengeance , I say Shodan wins .... you cannot play better games than System Shock 1/2 although with today's hmds , ghost free 3D, headtracking and LEAP Motion( my next quest) you can play them better.... Get over to GOG to pick up System Shock 2 , join/start the petition for System Shock 3 and 1 and the return of Looking Glass .. and my favourite Terra Nova Strike Force Centauri...
Both games ran fine on Windows 8 Pro so when Crytek have done melting your PC with Crysis 3, and Hideo and Platinum have occupied 6 hours of your time mashing buttons remember there are some great classics out there waiting ... Plus the more we buy System Shock 2 ...the more likely Looking Glass and Warren Spector will be freed from Mickey Mouse duty..... to code System Shock 3 ...! What goes around comes around....
The cool dudes may be messing with Unreal Engine 4 , but there is still plenty of mileage in the old Unreal 3 Engine yet...
With regards to testing games with the Cinemizer OLED headtracker it gives us two specific boons : Firstly the engine has been around so long that independent games developers are producing games that outshine Cliffy B and co and can easily be played in 3D using Tridef 3D. Secondly the use of the Unreal engine on PS3 means there are several games with native mouse control that we can now play using the Zeiss headtracker and any developer out there can add headtracking support simply by turning mouse support on for their Unreal Engine PS3 games ..
First up today let's look at a little play of Anti Chamber ( the demo ) using head tracking and also in 3D using Tridef 3D. The demo is a little hard to find but Google around and you will find it ! Anti Chamber is one of the finest games you can play at this point in time , on a static 2D LCD it looks pretty esoteric but on a 3D ghost free headtracking hmd like the Cinemizer OLED ..well ...welcome to the future .. This is just a few minutes of play recorded with FRAPS , all FOV movement is through use of the headtracker alone . In this example we just connect the headtracker to the USB port and go .
Now let's take a look at the PS3 games we can play using the headtracker . Starting with the DUST 514 Beta In the following video I connected the Cinemizer OLED headtracker to my PlayStation 3 which is connected to a TV without the hmd in order to film the action . You should be able to see the headtracker can easily operate the PlayStation 3 menus , jump into a game that supports mouse control and you can actually control your field of view .... When wearing the headtracker attached to the Cinemizer OLED hmd the gaming is far better , but the only way I could film the action on the PS3 is by taping the headtracker to the back of my head ..John McLane style...
I also finally dusted off the original PS3 version of Unreal 3 : the playing of which is aided by the head tracker as you tend to look around whilst playing this game even without the headtracker attached, so it feels far more natural to game this way. You have been moving your head around to look for years whilst playing FPS games without being conscious of it. Now that real world movement is reflected in the game world upping the immersion factor. When in game , headtracking is very cool when you ride that hoverboard , turning to travel where you look and in any of the ground or air vehicles , doing this with a thumbstick or mouse always felt artificial similarly looking down from an airborne Cicada targeting adversaries far below you on the ground just became far more impressive.
In the FPS sections , you will find yourself looking round at your enemies before you move your joypad , decades of console gaming have you used to instinctively reaching for a controller to have your avatar act in game .... you can still do that ..but now you can refine your movements .. you might strafe with the joypad whilst looking round the other way back over your shoulder with the headtracker creating an action in game that is quite impossible or unnatural to reproduce using a joypad...
Headtracking then in games is not about VR immersion , it is about refining and adding to existing control options , combining with them to produce subtle changes that most of the time are so natural you do not even notice ... an enemy flying by that you raise your head to look at , or a highlighted objective that you turn towards and so does your character or FOV ... you just performed all these in game actions without thinking or actually doing anything to input those changes through a mouse or joy pad...
None of the PS3 games are actually optimised for dof axes headtracking .. I would love to see the developers of DUST 514 add this option to their game ..... however emulating mouse input to give us fov control in 1000s of existing games on PS3 and Xbox 360 is something we can aim for..
My use of the headtracker with DUST 514 or Unreal 3 on the PS3 did not slow the games down in any way ...there are no overheads that you can perceive ...most of the time is initially spent going wow, followed by the realization that you have never really occupied these gaming worlds before you got your headtracker ..and why didn't anyone come up with this tech sooner... ?
My next mission should I decide to take it ( which I have) is to get mouse emulation working with all PS3 and Xbox 360 games , this can be achieved in many ways from Eagle Eye ( which failed) type hardware to GIMX ... Once I have that up and running I will be playing Halo,Rez, Wipeout, Child of Eden, Uncharted, Battlefield 3, Crysis 3 and more all with headtracking on console ... Well it will give me something to do whilst waiting for PS4 and Xbox 720 ..or the iPad 5...
If you are a console fan constantly fighting for gaming space , TV space or time a hmd like the Cinemizer OLED is heaven ...personal gaming without interruption .. anytime anywhere .. in the smallest corner of your noisiest and busiest room ... throw in headtracking and you will be a happy gamer indeed.
All of this is possible because of the ergonomics, optics, comfort and the ace in the hole headtracker attachment of the Cinemizer OLED hmd..
And as for PC games ... I dare you not to say wow when you try out a game like Anti Chamber in ghost free tridef 3D on the Cineimizer OLED with headtracking all powered by a single USB cable connected to your laptop ...
Yes there are people paying $16000 for GLASS, the whole VR world is resting on Palmer Luckey's shoulders ..hmd fans are as foolish and hungry as they come still trying to fit their HMZTs for extended Avatar 3D viewing sessions ..but you know you can get some pretty decent VR and AR gaming on right now with the very decent tech around you ... including the likes of the CInemizer OLED hmd + headtracker which you can take with you anywhere ..right now ..no jumping through flaming hoops or out of planes or swinging kids round to pay $1500 for a dev unit if the Google Gods select you ... hmmh that may be a little bit of bitterness on my part that GLASS is not available to UK hmd fans!
Why ....? because with the Zeiss headtracker we can....
How ....? PS3 sixaxis emulation lets us use a PC mouse to control PS3 games and as the Cinemizer OLED headtracker emulates a mouse, this lets us use the headtracker as a PC mouse and thus introduce headtracking control to any and ALL PS3 games , PS2 games , PS1 games which assign the right dual shock stick to fov movement ... and as I would discover sword swiping in Metal Gear Rising ..yes I really can slice a melon with a swipe of my head using the Zeiss head tracker... it's very messy though! We can introduce headtracking to PS3 games with software and or hardware... none of which requires hacking of any kind!!
1. UBUNTU + GIMX Bluetooth.. the first method ..... does not cost you anything either ..relying on stuff you have lying around ..say a laptop or Raspberry Pi and a USB stick totally free , and it will get you acquainted with UBUNTU ...
I downloaded Ubuntu.... made a live Ubuntu USB... and followed the instructions on the GIMX page to install it . The PS3 is blessed with Bluetooth controllers so we are going to use the Zeiss headtracker with Ubuntus Bluetooth recognition to talk to the PS3 .. which thinks the PC or Raspberry Pis Bluetooth signals are coming from a PS3 dualshock...how cunning... Advantages ..we are in undiscovered country here with the Zeiss headtracker , we know it emulates mouse control (as well as having dof axes) but we also know that using something like the Eagle Eye requires the Eagle Eye to recognise the type of mouse it emulates...which I found out it did not after spending too much money.... GIMX uses your laptop or your Raspberry Pi ...and therefore allows standard USB mouse detection simply put , it will parse signals from any controller you can connect to your PC USB port.. So your Ubuntu or Windows PC will recognize the Zeiss headtracker and so will your PS3 ! No worries about whether the headtracker will work ...it will do ! Promise.. GIMX has an amazing level of customization .. so we should also be able to tweak the resultant over responsive or under responsive head tracking control.. for each game individually...after all the rate at which you move your head or the way that you move your head will differ to how everyone else does. Translating or tailoring this individual movement to a headtracking action that in turn translates to an in game action is a challenge even the fastest headtrackers are going to face ( yes I mean you Oculus Rift) . Using GIMX we can tweak to our heart or heads content how that head movement translates into how much of an action in game ...for each game ! If you find your head movements are moving a sight too quickly or too far in Battlefield 3 ..just tweak the settings in GIMX...
About a day of tweaking (note to Sheldon :Sarcasm!) later as well as playing around with Ubuntu on my otherwise Windows 8 laptop , after discovering Ubuntu loves touch control and the headtracker as much as Windows 8... I stumbled on a solution to the lag problem I was having ...I turned off the force update and subposition tabs , set the dpi to 2000 and suddenly I was whizzing around in Crysis ...almost lag free... then it all started to click and suddenly I was playing any game I chose using headtracking..on a console...
2.Windows 8 + GIMX Serial + Cronus : The second method I tested required the purchase of a Cronus device ...
Using GIMX Serial from Windows 8 with a USB lead connecting the Cronus to the PC I had 0 lag or setup issues... in this instance I had no problems just jumping into Crysis ..leaving all settings like Forced Updates and Subpositions checked
You can also build your own diy USB adapter for around half this price ..but I also wanted to test if the Cronus would take a direct connection from the Zeiss headtracker ... which although it does not out of the box ..it may do after an update or a little bit of config editing ... the same way Eagle Eye and XIM should also work if they were told the id of the mouse that the Zeiss Cinemizer OLED headtracker emulates... but you cannot get at them the way you can get at Cronus through Gimx...
Cronus has a side micro USB to USB port that we attach to the Laptop and therefore Gimx
Including Gimx in the loop has serious advantages, mainly being able to tweak configs for each game or game type..as we chose and not having to pay,wait for or download developer approved config files... the way you want your sights to move in a FPS game may not be the way you want the FOV to move in a third person game for instance ..
At this stage though I was overjoyed to get head tracking working in ALL PS3 games and this solution means we can get headtracking in ALL PS1 and PS2 games too....
and here's the proof:
Remember you do not need an expensive laptop, or desktop, you can achieve the same result with a Raspberry Pi.
The most fascinating prospect though is that of mobile computing. Our computers are getting smaller,I used a laptop in this test imagine doing the same with a wearable computer, our phones are about to run Ubuntu ... at a very near point in time you could have your phone or mobile pc allowing you to use headtracking on your console games or PC games ...wireless... And not too far off we could have the successor to Bluetooth or Wifi IN the hmd or headtracker ...We are closer to wireless headtracking and wireless display hmds than we think... it's not all GLASS! I was not expecting the lack of lag I achieved with GIMX Bluetooth .. but it worked !
What ...... happens when you start to play PlayStation games with headtracking on your Cinemizer OLED hmd??
The above video only shows you proof that you can use the Zeiss headtracker with any PS3 game and cannot convey the experience of actually playing wearing the Cinemizer OLED hmd and using the headtracker for the very first time to play PlaySation 3 games.. but that's what I'm here for... With the advent of new control schemes like the Zeiss Headtracker and LEAP Motion ( more on that next time) reviews of games are going to involve HOW we play as much as what we play .. for instance
FPS games this is where current headtracker development ( Oculus Rift ) seems entrenched ..with the Zeiss though we can play FPS games on our PS 1,2 and 3
Crysis PS3 ..the best in the series and always the first game I needed to test head tracking ... Lush Cry Engine 3 jungle and sand stretch out before me and with the headtracker I can finally just look around all of Crytek's creation ..down to the small birds around the different types of foliage , directly at the gunfire coming towards me ..cloak engaged ... The whole way you move through the environment changes with head tracking ..you look round just like you would if you were traversing real terrain. This is subtle so much more different than the old now legacy mouse or PS3 right thumb stick .. after all what is the beauty and power achieved by Cryengine 3 ..(!) when you have to reach down to a mouse or hit a thumbstick to look around at some tweety bird thing or the shiney alien tentacles about to grab on to you ... why can't you just look at them and so does your game ... wait you can with the Zeiss headtracker...
Third person games : third person games are going to benefit greatly from headtracking fov control .. as much as FPS games will...if not more...
In Journey with the headtracker taking over the right stick duty .. we can now move our field of view and run towards where we are looking .. And that beautiful award winning world of sand can be experienced in a whole new manner ..using the headtracker to spin round the view or move to a point is jaw dropping . HOW you play just changed before you would shift the right thumbstick to a point of interest wait for the fov to swing round and then push the left thumbstick whilst holding the right stick in the direction of where you want to go... With the headtracker you just look towards where you want to run , pushing the left stick (or in this case keyboard) forwards to run there ...
There is a feeling of awe and connection to the game world as not only the screen moves to your head movements but your character runs and slides down sand dunes following your head movements..see something interesting you no longer push a stick to get your character running you just look at it and your character is already heading to wards it , see something interesting to your left just turn to look at it .. I find myself sitting on a dune just looking around using headtracking at the serene landscape ..my head movement effortlessly framing the beautiful vistas
Metal Gear Rising : as I said earlier theoretically you can mash melons with your melon! Yes just shake your head and he will shake his sword...totally useless ..but you can also control your fov ...
Resident Evil 5 ... I am walking that street at the start of the game and looking round like I want to ..and so is my character ..it feels much closer to being there... and so much cooler than using a Move or thumbstick to change the fov...you no longer have to keep a button or thumbstick in a certain position to look at something .. you just look .. Then there are the obvious driving and simulation games ... all waiting to be tested! So we now have Carl Zeiss Cinemizer OLED headtracking in any PS3 , PS2 and PS1 game...
Ask not what Virtual Reality can do for you ask what you can do for Virtual Reality....This seems to be the mantra behind the burgeoning VR scene..and I wouldn't have it any other way ...
I have been using hmds now for almost two years on a daily basis, watched them evolve from the headcrab that was the HMZT1 through the ST1080 to the the Cinemizer OLED and my soon to arrive Rift ( come on Palmer keep calm and carry on!) .Hmds are only part of the puzzle that is VR .. Another piece is how on Earth we each personally interface with VR worlds,to do so we have to let go of legacy controllers like joypads ,keyboards and move beyond them assuming someone has managed to create those worlds ..which is yet another piece of the puzzle..
the moving finger moves......
We also have to move beyond the current obsession hardware developers have re headtrackers , hypothetical lag that exists or does not exist on pre release hardware or in software ... that itself does not exist outside pre alpha code..For VR to take off , it will take more than a fast headtracker hard soldered to your hmd circuit board sorry app devs ... For VR to take off the end user expects lag free headtracking, voice recognition, hand, palm, finger, thumb recognition ,the ability to include or add on legacy controllers and most importantly the ability to use all this with not only promised VR apps ( launching whenever they are ready) but with ALL the thousands of apps and games that we the users have spent our lives accumulating..
and having moved so does Kratos......!
The problem of lag/latency if it is discernible when using your headtracking hardware is a problem that exists in that current hardware, to solve it either we wait for the technology to catch up to the idea of lag free headtracking and use brute force ..or with current headtracker tech we offload the grunt work to the cloud or other hardware,which we can also use to tweak, accelerate ,adapt ,adjust or in some cases even slow down the headtracking.. The developers of Oculus have to be careful they do not talk their device out of existence with all the scuttlebutt on latency that is clouding the launch of the developer kits this week ...yes I have one coming too !
it's full of stars......that you move !
I have used headtracking now for 3 months . never experiencing latency or lag in any scenario , I have used LEAP Motion now for over a month and again never seen any discernible lag . In fact if you look through my diatribes on the Cinemizer OLED headtracker and the LEAP Motion I often talk of these devices being over sensitive and that is on what most core game developers would think is obsolete hardware...
I also do not have much time for the idea that VR hmds should only run specific VR ready code.... I always intended to use my hmds to interface with any form of display, OS, game , app or kitchen appliance/lawnmower ..I need to shoot latency free Force lightning from my Sith hands now ..not when Senior Carmack licenses the engine for Doom 5 to Gearbox or Rovio(laugh but it will happen) and as you will see I can ...
The ability to shoot Force ligthning from your hands pales into insignificance compared to the true power of multiple combined latency free VR control input mechanisms
errhm ...with all this in mind I set out to use my i7 X220T running Windows 8 to combine the Cinemizer OLED headtracker AND LEAP Motion finger tracking on 10 year old hardware .. namely the PlayStation 3 and Xbox 360 ... So we are going to use actual games , play them with multiple VR leaning control methods 1. Zeiss headtracking 2. LEAP Motion finger tracking 3.Kinect voice recognition and throw in some legacy control too ...
Note I would use the Cinemizer OLED hmd to play these games in 3D and or the smd ST1080 ...to show you it works though I am filming off a normal home Plasma tv...!
Are we ready ? then let's see how we can combine headtracking , voice recog and finger tracking to play Xbox 360 games ...any Xbox 360 game ... and PS3 games ...any PS3 games...
1.Xbox 360+ Cronus+Bullseye + Win 8 running Bullseye+Dangermouse + Leap Motion for finger tracking +Zeiss Cinemizer OLED headtracker + Kinect connected to the console
Xbox 360 gaming with VR leaning controls :
Halo 4 with LEAP Motion finger tracking and trigger action combined with Zeiss Headtracker FOV movement
Halo Anniversary lets us combine Kinect Voice recog , Leap Motion finger tracking . Zeiss headtracking AND play in OLED ghost free 3D on the Cinemizer OLED hmd...
Ikaruga played with LEAP Motion with the thumb click mapped to the trigger...my score's improving already....
Combining LEAP Motion+ Cinemizer OLED headtracking to play Xbox 360 games : Hand tracking combined with headtracking to play REZ HD,Panzer Dragoon Orta , Ace Combat 6, Halo Wars, Shenmue 2, Bastion, Radiant Silvergun
this may look good ...but it feels totally awesome when you play using the LEAP and or Zeiss Headtracker!
Panzer Dragoon Orta with LEAP Motion control NB we could easily map the FOV shoulder movement to the Zeiss Headtracker too
This is an example of where headtracking does not need to mirror real world head movement ..in a game .. Panzer Dragoon Orta uses rigid shoulder pad view shifts ..we could map these to left right turns of the Zeiss headtracker , and or we could also map the entire FOV control to the headtracker , whilst we use the LEAP Motion to handle the targeting, weapon selection, or movement..
Rez HD playing with LEAP Motion ... now you can feel the vibes from the 360 controller in your pocket ..whilst playing the game with Finger tracking ...Matron!!!!!
look around or pilot your Ace Combat 6 planes with your fingers,hands or your head!
Before Ikaruga there was Radiant Silvergun...here played with LEAP Motion finger tracking ...I have yet to figure out what to do with the concurrent headtracking!
Combining Leap Motion+Cinemizer OLED headtracker+Kinect Xbox 360+Legacy control :
This is very nearly the VR control Holy Grail, using Mass Effect 3 :which lets us combine 3 VR leaning control methods : Kinect Voice Recogntion, LEAP Motion finger tracking and Zeiss Cinemizer OLED headtracking ... Note we are actually using 4 control methods simultaneously if you count the legacy Xbox 360 Pad.. If you really want to push it you could have Kinect on the PC too , with full body tracking in addition to voice, head and hand recog! (but that is for another time !)
You should be able to see there is 0 lag that can be discerned by the user when combining LEAP Motion and Zeiss headtracker use on Xbox 360 games using this solution. And that we can if our games allow it also add voice recognition to the mix. The Cronus and Bullseye combination is most impressive , there is no screen judder or delay when using our VR leaning controllers with this set up . I found it simple to tweak settings from Bullseye too . Lastly we can see that LEAP Motion runs parallel to any other form of controller that we want and we can use apps like Dangermouse running alongside them. So on to PlayStation 3 gaming with VR leaning controls ... PS3+Cronus+Windows 8 i7 Lenovo X220T running Bullyseye+Dangermouse+ Leap Motion +Zeiss Cinemizer OLED headtracker +/- any other input we can connect to the PS3 or X220T!
Some of these controllers are not like the others...
Combining LEAP Motion+Cinemizer OLED headtracking on PS3 to play Child of Eden
How cool is that? Now imagine..well you do not need to imagine..just attach the Zeiss Headtracker to a Cinemizer OLED or ST1080 hmd and you can do all the above in OLED ghost free 3D ..or LCOS 1080p ghost free 3D... right now.
playing Child Of Eden in ghost free OLED 3D with LEAP Motion and Zeiss headtracking is simply mind blowing!
Combining LEAP Motion + Cinemizer OLED headtracker in PlayStation 3 games test 2
Combining LEAP Motion+Cinemizer OLED headtracker in PlayStation 3 games Metal Gear Solid 3 , Metal Gear Rising , DUST 514,Uncharted 3, Crysis , Wipeout ,DYAD all tested with combined LEAP Motion finger tracking and Zeiss Cinemizer OLED headtracking
you spin me right round ...
right round....!
both Zeiss headtracking and LEAP Motion working in conjunction ...
you know for when Raiden goes to the dentist...
LEAP Motion and DangerMouse,Bullseye and Cronus used to control Cryengine 3 Crysis on PS3 you should experience this in 3D!
Zeiss headtracker and LEAP motion used together to play Crysis on the PS3 ...no conflicts , no slow down ... no discernible lag
Using the Cronus both Cinemizer OLED headtracker and LEAP Motion can be used to play DUST 514 , being able to tweak settings from Bullseye also means we can do far more than with direct connection of a mouse and keyboard to the PS3
With Dust 514 I have already spent time using the the Zeiss headtracker directly connected to the PS3 , taking over the mouse FOV duties. With this new solution though we can do far more .. first we can set up both hands to control the game , in the video I had time to set the LEAP to the left thumbstick , which let me move the character and also to the FOV stick which let me move the field of view .. The Zeiss headtracker ran alongside the LEAP without issue .. this means we have plenty of VR leaning control options to combine and play DUST 514 .. what we are doing here is looking for and at new ways of playing.... moving your FOV around quicker than the next guys headtracker is not going to be enough to kickstart the VR gaming revolution...
VR gaming control is here ...
all that stuff you saw in movies like GAMER ...
it is in your hands ...
and on your head too if you are using hmds...
There is 0 lag or latency when using the Zeiss Headtracker and LEAP and both of them together to play MGS3 !
Snake responding obediently to the Zeiss headtracker ...as will Gabe .. and Sam too !!!
Using LEAP Motion to turn on my shield and the Zeiss Cinemizer OLED headtracker to steer and accelerate ....
What you see in the above pics are the subtle movements to the mm that LEAP Motion can pick up. Note though, we do not need all this power , what matters in making LEAP and Zeiss headtracking viable VR controllers is how we tweak those small movements and convert them to ingame actions that the gamer and not hardware developer with a slide rule expects ... We can get at this using the Bullseye software, the LEAP sdk, DangerMouse and the Zeiss headtracker sdk. But I purposefully did none of that .. because I want to come at this from a gamers point of view .. they want to connect these devices and play ..not spend hours updating drivers .. Yes you can on the developer side do all those things , but remember the Wii Mote took off not because people were measuring or adjusting it's lag but by the simple fact people were able to pick up and play . I have to say using the Zeiss headtracker and the LEAP is actually simpler than the Wii mote .. no drivers , no lengthy calibration routines ...
As you can see there is no end to the possibilities of playing games with VR leaning controls and combining them, in Uncharted 3 I would be playing in 3D using the Cinemizer OLED hmd , moving the FOV with the Zeiss Headtracker and moving the character with LEAP motion .. in the video I have just the punch button mapped to the thumb click. LEAP Motion does gestures too, so a punch could actually be a punch ..steady on though we do not want to go the KINECT route ....!
DYAD is one of the simplest games to demo LEAP Motion with , the response is excellent 0 lag!
Leap Motion used to play God of War Ascension
Whilst I was errh testing all this out, my God of War Ascension arrived and as I had everything set up and with the demo already installed,I thought I would give playing it with LEAP Motion a go ... And I was pleasantly surprised, these games tend to twist your thumbs off after a while so mapping one thumbstick to the LEAP was a revelation, suddenly I was jumping , and combo fighting with a twirl of my finger or hand movements or a click of my thumb. More than just a test , I found I was actually playing through a level .. using a VR leaning control method to play what is essentially (the epitome of) a side scrolling beatemup.. Compared to the PS3s own motion control The LEAP is insanely fast even at this developer stage. Note you could map both hands, 8 fingers, 2 thumbs, add voice recog for other commands using the method here .. Looks like I am finally going to finish some of my PS3 games ... VR style ... We have not even begun to tap the potential of the LEAP and Zeiss Headtracker !
Here is our test setup ... should you wish to try this out for yourself... 1.Xbox 360+ Cronus+Bullseye + Win 8 running Bullseye+Dangermouse + Leap Motion for finger tracking +Zeiss Cinemizer OLED headtracker + Kinect connected to the console
2.PS3+Cronus+Windows 8 i7 Lenovo X220T running Bullyseye+Dangermouse+ Leap Motion +Zeiss Cinemizer OLED headtracker +/- any other input we can connect to the PS3 or X220T! Note you could connect the Kinect to the laptop and add full body tracking to your PS3 games using this setup...
Theory and Method:
They say you get lag and latency issues on the fastest of hardware , so I set out to use up to 10 year old hardware .. the PS3 and Xbox 360 to show otherwise...note even with direct connection of the Zeiss headtracker to the PS3 there is no discernible lag in any game you play.
The Cronus lets you use mice and other input methods on either console , the flaw is it has to be connected to a laptop or PC, allowing adjustment and user defined control setups through Bullseye software ..
But that flaw is just what we need to connect our VR leaning controllers and to take the load off the consoles looking to eliminate lag...the consoles do not have a clue you are using finger tracking to play God of War Ascension or Halo 4 ...but hey that's consoles for you ...
Now since the Zeiss headtracker emulates mouse control this gives us headtracking in any game we choose on the consoles as I have shown before with PS3 ,PS2 and PS1 games and with Xbox 360 games ...and yes it does the dof axes too but we wont be using them here
True VR is going to require more than just headtracking ...we want to use our fingers thumbs and palms ... this is where LEAP Motion comes in ..erhh handy...
But how do we get it working with the Xbox 360 and PS3 ...we need the Bullseye to see it as a Windows controller ...say something simple ...like a mouse for instance...
DangerMouse is a java app that runs from a command line and gives you mouse like control using LEAP Motion only using your fingers in air ...
I had the idea of running this in parallel to the Bullseye software ..and it worked like magic ... the Bullseye and Cronus see the Zeiss headtracker and the LEAP Motion as Windows controllers ..mice in this case ..which through Bullseye we can easily map to any function of the Xbox 360 or PS3 controllers..
As the Cronus turns any controller you connect to your PC into a PS3 or Xbox 360 controller ..we can have multple VR inputs .
We can have BOTH headtracking and finger tracking control in ANY game ..
But that is not enough for VR we also want voice recognition and this we achieve through Kinect attached to the Xbox 360 .. Now the poor old ageing console can barely manage full body recognition but we don't care about that for we have the power of LEAP , what the Kinect can handle well though is speech recog ...
And thus we are ready to test not one but 3 forms of VR control , that will be essential for VR to take off in the consumer arena on aged hardware along with legacy control too..
If this seems a long way around, it is , deliberately so ... I want to introduce points of latency into the control loop to see if a) we can use 3 VR control methods together b) if there is any discernible latency or lag c) Asymmetry my dear Watson is the future ...using the cloud or a higher powered device(our laptop ) to do the grunt work for a lower powered or closed device(our PS3 and Xbox 360) that could not handle it ...and most importantly d) So I can stop asking William Gibson are we there yet?
I could also have substituted GIMX for the Cronus and used a wireless solution ( but that is for yet another day)
So developers, take some time off using your slide rules , grab your favourite game and gaming device , take some time to actually test out all the multiple forms of VR leaning control . Actually play something and then go back to the lab... If the development of VR continues along a path where all talk is dominated by problems, problems, problems we may just miss the simple and obvious solutions staring at us ...
I have not even begun to tap the potential of this way of playing games, more to follow !
And good luck with the launch Palmer and Co ! Thanks to Zeiss for the headtracker ,LEAP Motion, DangerMouse , the developers of Cronus
.JPG)
and in the USA the 
, .PNG)
.PNG)
.PNG)
.PNG)
.JPG)
.JPG)
.JPG)
.JPG)
.JPG)
.JPG)
.JPG)
