For those of you who have yet to try the Daydream, its mostly a really nice Cardboard with a higher spec phone for better rendering and head-tracking.  (I talk about this in my previous post)  But what is really different, and IMOP better, is the controller. Its not the design of the controller is all that great, its the fact that its there! Having this sense of presence both from the haptics of my hand as well as my head movement greatly improves immersion and enjoyment.  The presence of a controller my not seem like a big deal, but it is.

What is really great about the Google Daydream is the commitment of a controller.  When you buy a headset, it comes with one.  Thus app devs can always rely on the fact a controller will be there. This certainty is what we are missing today.  And Samsung and other other headset makers can take cue and make it a requirement for their systems as well.  Unfortunately, as I understand it the Daydream controller will NOT work on other systems, but look for others to fill this market.

Also coming is the mobile stand-alone headset.  I do not expect to see anything beyond POC at CES2017, but I fully expect that towards the end of 2017, we should see a few come out.  These devices will have controllers as well, allowing devs to leverage previous work on the Daydream and GearVR systems into the new stand alone systems.

Also watch for 6DOF controllers as well..  This will be a bit harder to crack.  

Technical

The obvious thing is technically what they pulled off.  Using multiple xbox sensors (I’m not sure how many) they are able to do full body tracking.  Because the user’s vision is completely obscured by the HMD, they can place the xbox sensors and light it to provide the maximum effectiveness for body tracking. Hand and head position most likely are sent via closed wifi to the gear.  From there its rendered in (Unity?). As you can see in some of the shots, not only is head positional tracking going on, you can see your hands on the rope as you walk across!

This is great. Now add wind, cold air and sound (notice there we saw no headphones so the sound was most likely piped in), and you have a very deep immersive experience,  Because the Gear VR is wireless, the user is not restricted by cables.  In fact, once you are immersed, and given all the other sensory input, your brain can have a real fight deciding what is real and what is VR.

Even better is the fact that you put on the HMD outside the room, (again as best I can tell) the user never saw the real bridge that was constructed, so the bran has no other cognitive interpretation of what is going on except for what they see. BTW, don’t misunderstand me, with exception to perhaps one person, they KNOW its a simulation and short of flinching and perhaps lowering one’s body, they are enjoying it for what it is.

Marketing

I would guess most people who watch this video, especially those in the VR world would mostly be thinking about how great of an immersive simulation this is, and move on. But what really caught my attention, and what I like and admire just as much about this concept, is how well this works from a marketing point of view. Remember, they are releasing a (new?) instant latte. The sell point for this product is you can enjoy a cafe quality latte anytime, simply by using a microwave.  Ok, so what are the customer scenario where they may want this?   Most likely at home and perhaps the office.  I think most offices offer coffee and if not, typically there is a cafe near most large office complexes.  But at home, not so much.

In the experience we saw in the video, you are asked to cross this long dangerous bridge on a distant exotic mountain top in a blizzard!  Many dangers face you on your trek to get to the other side where a microwave waits for you.  Each step on this old decrepit bridge becomes more dangerous as the winds increase, and rocks start to fall all round you!  Somehow, through perseverance and a bit of luck, you make it to the end where, you find a microwave, (yes people do randomly leave microwaves in the Himalayan Mountains, just go with it). You open the door and a wonderful hot beverage is waiting for you to warm you up, body and soul.

Now, think about a long snowy winter train ride come where you have to walk the last 3 or 4 blocks.  Not too different eh?  And if you like lattes then knowing you have a nice warm drink waiting for you could be that much better.  So the sell point here is to associate a tough trek with a warm reward.  This experience drives this idea to the customer is a very real and tangible experience. An experience they will not soon forget, and share with their friends, including the idea of the microwave latte and the branding that goes with it.

Execution

I have not experience this but having been doing VR for so long, and having tried everything from the CAVE to the wind bird simulation, its not hard for me to put myself there and have an idea of what this would feel like.  I love the fact that this experience really allows you to ‘walk’ through it.  Yes we have plenty of simulations where you can walk, but typically its flat ground.  But here you have a bridge, you have rope to grab.  As the wind starts to push you back, the IMU on the headset reflects the sublet changes in head position because of it.  As you walk, the scene walks.  With the xbox sensors and the very well lite room, body tracking can be very precise.  I’m sure a dedicated local wifi was running allow the fastest possible update time that can be done with current technology.  Based on the headset graphics it looks like Unity is running this, but again, because of all the various sensors being reinforced with the same cognitive interpretation; you are on a bridge, in a cold wind storm, with the proper sounds, the less than realistic graphics does not matter so much. For some people, its too much!

BTW< this could have been staged, not sure.

A small note here.  Notice no gloves on peoples hands.  They are asked to walk through this cold experience with no gloves.  What that means, and you can tell at the end when people come out, is that they are using the drink to warm themselves.  A GREAT bit a experience detail.  Even after the HMD is off your head, YOU ARE STILL COLD! The experience is still with you, and the reward of the hot latte continues to reinforce the benefits of buying this product.

Appropriateness

So here I want to reinforce something about VR and marketing.  Like I said, I have been around VR for a long time, 20 years plus.  I have seen MANY marking events that try to use a VR experience to create an association of “high tech” or “cutting edge” or “next generation” to a product, some successful, some not so much.  In all cases however the VR experience has little to no association with the value of the product. Further, once the VR experience was over, that was it.  I typically remember what I liked or was annoyed by in the experience, to date I cannot remember a single product from one of those marketing campaigns. This is different.  Even though I did not experience it myself, the whole idea of walking through a blizzard to be rewarded with a hot latte seems to stick.  As the more real the struggle to reward is, the more likely you will have a favorable memory of the reward. What is appropriate about using VR here is that VR has nothing to do with campaign.  It is simply the best tool to achieve the maximum positive reward association with the product. Not trying to be ‘new tech’ or ‘cutting edge’ or ‘cool’.  well ok, a bit cool.  But instead simple marking 101, create a positive association with your product, and VR in this case, was a great choice.

Now, check out the Video.

I took two plane trips in the last 2 weeks and decided to see what level of utility the gear VR would be on a flight.  Not only was it great, it totally changed the flight experience. In this post I’m going to list my favorite VR apps for flying and WHY it was so much better.

First, lets set the stage. Most of the time when I fly, I do not get to fly first class and often I’m sitting behind someone. short continental flights means I’m crammed into a small seat, with little leg room and even LESS room for my laptop if the person in front of me decides to recline back… which is often.

So here we are in the little space, unable to work comfortably and if you want to watch a movie you have those little screens in the headrest in front of you or worse a large screen somewhere on the ceiling.  Not the best experience.

Now lets introduce the Gear VR to this situation.

We already know that by putting on the HMD we would find my ourselves immersed in a virtual reality.  Whats great about that is almost all the VR immersion take you to a space that is FAR larger then the little seat you are sitting in.  Add good headphones and you are fully isolated from crying babies, people opening and closing blinds, and the person in front of you trying to recline into your lap.  Even though I knew this would happen, the effect and relief was beyond expectations.

Although the Gear VR is not perfect, and you have to deal with cooling and power issues, I found I was able to stay in the HMD for multiple hours per session.  On my flight to Spain I watched Star Wars IV.  Using the oculus theater mode, I started the movie using the cinema setting, making me feel I was in a large empty space.  After about 15 min I switched to ‘void’ mode and did not lose the feeling of watching a large screen TV in a large comfortable setting. BTW, helpful hint here; to maximize the battery live of your s6, turn down the brightness, stay in airplane mode and use ‘Void’.  The void setting will help reduce demand on the system to render all those polys that make up the various movie settings.  Also, turn on the air and have it blow on you.  It will cool the phone, and you, and help with fogging.

One issue, well two issues I’ve found is the movie tends to drift over time and I have to recenter it.  Annoying.  Also when the plane turns I have to recenter.  Its not a big enough issue to stop me from watching movies, but hopefully they will find a fix for this.

A common question I was asked is; can I play any movie?  Yes.  As long as its in a mp4 or other supported format, you should be fine.  If you place the file in you ‘movie’ folder it will find it.  You an use MilkVR too.  Create a folder called “MilkVR” and it should find it.

I preloaded a few movies, some TV episodes and few random video’s I shot. Worked great.

Now on to games.  Here you need to be a bit more selective.  Remember you are sitting in an airline seat and can only turn your head somewhat.  ALSO remember you might have someone sleeping next to you so lots of turning to see 180, which means you have to often turn your shoulders could be out of the question.  So get games where all the action is not more then say 160 fov.  My absolute favorite game, Bandit six o’clock, is around 120.  Perfect.  So if needed play your game a little before the flight in a ridged chair.  Many games lose their essence if you are not in a swivel chair.

Next, try to be sure your game uses the gamepad.  If you game only works on the HMD trackpad, then either make sure you get the window seat or the person sitting to your right is quite tolerant because odds are you might smack the with your elbow.

Finally, spend the money and get full games.  Remember, you are trying to kill time up there and a demo is not going to last.

There are a few games I like but NOTHING come close to bandit six o’clock, and even MORE so when flying.  The game is simple.  You are a tail gunner on a WW2 Lancaster.  Enemy plans are trying to get behind you to shot you down.  What is so great about this game is the game play is actually a bit more sophisticated then it sounds.  Also the immersive effect is amazing for a game.  Unlike most of the games I play I really sense the dimensionality of the airplanes when they are flying in.  Its even more believable when you get a bit of turbulence on your flight!  I played this game for 1.5 hours TWICE! and for an hour on the Spain trip.  I only stopped because of the phone overheating or I was interrupted for one thing or another.

Now some tips.

* Plugin earphone or nice a nice headset.  BRING it! shutting out outside sound is your friend.

* Bring back up power.  Although most flight has AC or DC, not all of them do.  Be prepared.

* Bring a cigarette DC adapter.  My section had a DC socket; I got lucky that I asked the flight attendant right away and got one. I may not get so lucky next time.

* Keep your jacket on and turn the air full blast.  You need to keep the phone as cool as you can.  I have actually hacked a small cooling fan for my Note4 and working on a s6 version now.  I’ll post the these modes when I get time. This is really important, the hotter your phone get the less efficient everything is, including the battery, and at 35,000 feed you need to save power where you can.

There are a number of other apps I played on the flight I did not talk about but the main point is the VR helmet as an escape from a small airline seat was extremely successful. As such I will never fly again without my Gear VR.  And if you have a fear of flying, or hate small places, this is for you!

So this year will be just as good with even cooler prizes. Samsung is giving away a GearVR HMD WITH THE PHONE!! That’s right, win this category and you are set.  Of course, your team will have to figure out who actually gets the device, but that’s your problem.

Unless you have been living under a rock (and given the property values in the Bay Area that just might not be a bad idea), VR is the new hotness and everyone is in the game: Google, Samsung, Facebook (yes, Facebook!) Sony, HTC, etc…  The HMDs are more powerful, cheaper and better than before.  Tools make it easy to create PC or mobile VR games and apps.  Content from 3D video to full worlds is starting to be created.  We are still in the beginning, so VC money is still flowing to start ups that can find their nitch, and this is the time to get a team and try a hack.

Plus, it’s just plain fun.  

Go here to check out the details and get signed up.  There are lots of categories so plenty of ways to win a hack.  And VC’s love hack winners. Oh, and the VR Hackathon is offering free demo space on Friday nighty for anyone in the community who wants to show off their stuff!

A final note about the Gear VR and the lenses fogging up.  Every time I put this one the lens fog up on me.  To fix this I went to a sky shoppe and bought defogging liquid for the lens.  It works great for one use, but once it drys, then you are back to where you started.  This is a first try at a VR HMD for Samsung and they made it clear this is a developers version, so I’ll give it a pass for now.

Oh and use earplugs.  Worth it.  Many of these demos lose something when playing sound through a speaker.

Unlike other sites, I will put the list down here first, then I will create a page per app because each one make the list for a different reason.

10.   Playhead

I was really on the fence with Playhead because it did not appeal to me, but I appreciate the effort and the imagination that went into this effort.   I’m sure this would be higher on other peoples lists.

9.  DarkNet

8. Titans Of Space

7.  Vanguard V  :  Nightitme Terror  : Lunsee

6.  Dreadhalls

5.  Anshar Wars

4  Herobound: First Steps

3. Matterport VR : Gyeonju VR Museum

2.  Esper

1.   360 Video’s .  BTW the NBA videos look good but still wish the video was a bit more HD.  In a few weeks we are suppose to be able to watch the All Stars game.  Not sure if we get to see the whole game, or highlights or what.

Temple VR did not make it because they violate a basic rule in VR, do not move a users point of view too fast.  When you dodge from one side to another you will get VR sickness pretty fast.

First off I’m going to focus on VR and not include AR because although many people like to mix these two systems, immersive AR is VERY hard while simple HUD level AR is already available in your car.  So lets see what we need for a good VR system, which is basically the same thing that Ivan Sutherland created in 1968 (that’s 46 years ago kittens). His head-mounted three dimensional display contained most of the basic elements we consider needed for VR.  A head mounted stereo display, motion tracking, orientation tracking, high speed image rendering, HCI input.  Missing was 3 things from the ’68 system. Easy content creation, user comfort and finally the Internet.  But to be fair, the Internet did not exist back then, but I bet if it did Ivan would have pulled the content over the wire just to be cool.  Oh and one more thing that was missing.  Not everyone could afford to build one of these things.  As such, not much happened with VR in a non military theater for awhile although research continued on.   Then two things happened.  The mid 80’s and mid 90’s.

In the early 80′ Graphic workstations were cheap enough and powerful enough that professionals could buy them.  This allowed the general public to have its first taste of 3D generated images that were at the time limited to Hollywood movies like Looker and the landmark Tron.  These workstation were still pretty expensive but as each year passed, they got more powerful such that you could render over 15 frames a second and create the illusion of motion.

As we drifted from the 80’s to the 90’s workstations were getting more and more powerful and cheaper because of Moore’s law.  This allowed more and more people to play around with high pipeline graphic computing, so much so, you could actually render a textured scene over 15 frames a second!  3D became not something only done on very powerful and expensive powerhouse workstations, but on a workstation you could by in the 10’s of thousands of dollars.  Now 3D graphics and effects were all over television instead of just high end movies.  It was only a matter of time before young hackers starting putting the output of these workstations onto two small screens on a headset.  Using various approaches to head tracking, and suddenly VR seemed very real with store fronts allowing users to enter a pod, don a 3D headset and play 3D games. Siggraph got more and more interesting every year and each year seem to surpass the previous.

By now VR really took off in the public imagination. Films like Tron and later Lawnmower Man that introduced 3D and VR to a game called Dactyl Nightmare by company called Virtuality that when hit the Press made everyone go nuts. Seemed like VR was really here!  Now everyone was writing about it and people wringing their hands that this would be the end of our humanity.  But this was just the beginning, Moore’s law and more accessible 3D creation tools like VRToolKit and 3D editing tools kept the hype expanding… and I was there.

Back in 1994 I was doing research on VR through a service called Gopher then later using something very new and cool called the worldwideweb which you accessed through a browser called Mosaic (we are going way back kiddies) I started researching 2D cad, 3D cad, and finally VR while studying Architecture at UC California, Berkeley … Even in my own lab in Berkeley, I had a VFX1 HMD running on a PC.  A PC!  The leaps and bounds of computer power seemed limitless.  And with each new CPU came faster and faster framerates.  Memory was going down too, so putting textures on objects was not only a real possibility and broke real time ceiling that was limiting 3D with only phone shading.  Further the gaming industry had its first big break through. Quake! (we can argue duke nukem another time)  In fact, I found Quake was faster then any of the 3d rendering engines from any school at the time.  I did my masters thesis using modified maps with Quake and the VFX1 HMD.

And then… All went quite.  Was it over hype? Tech bobble burst? Wasted dollars on projects that never made sense in the first place?  Yes.

Fast forward to 2014.

So here we are.  The Oculus rift which was a geeky kickstarter project exploded and then got bought by Facebook.  It finally seemed that perhaps VR for the masses could be possible.  Then shoe No.2 dropped. At Google I/O a cool little project was presented called Google cardboard, which basically gave you a simple VR headset made out of cardboard using your phone as the display and computer.  Didn’t take long for a number of higher material devices to be created on the same concept including Samsung’s Gear VR using the Note4.  Suddenly we had the basic building blocks of Cheap VR for the masses.  We also have more content.  3D video is part of the VR offering making VR not just the real of playing video games.  Speaking of games, Unity now comes with an Oculus publishing option.  We also have broadband.. well most of us do.  So is this?  Will it really happen this time???

Ummm.. Not sure..  There still are some problems.

First, there is not THAT much content.  I see a number of videos coming out but they are all proof of concepts and many of them are not that good of quality. Games?  Yes there are a few games made especially for the VR experience but most are demos.  There is still more money focusing on the PC or console gamer.   VR sickness?  Its getting better with faster frame-rates and better tracking, but many VR designers still do not understand just how close to the edge the user is before they get sick of orientation disruption is is taken too far.  That being said…

I think there is a good chance we have hit the tipping point where VR can become a nitch consumer industry.  Right now I’m waiting with baited breath for the NBA All Stars 360 Video to come out for the Gear VR.  I’m really hoping they are taking the extra time to make it look nice and feel immersive. Speaking of the Gear, I love the fact that I can put the headset in the car or backpack and with my note 4, access VR anywhere, anytime.  THIS IS NEW and this is big.  With a bluetooth controller, I pretty much have all I need for a sit down VR experience.  This is something we have not had from the beginning.  In time I expect the HMD to shrink to glasses size again using the phone for computing power.  Its this freedom and flexibility that is what will keep VR from busting again.

Post:  Its March already and I’m doing a quick edit and posting this before I forget again.  In the time between first wriring this and today we have seen a number of new HMD’s come out as well as Microsoft’s amazing HoloLens.  Its the HoloLens I’m most excited about.  The usecase for AR displays in the home and work is mind bending.  I have not seen it in person, but believe me, the moment I do, I will be posting!!!

So my bet on VR/AR?   I do not think we have seen the full potential yes, or anything close, but I do not expect to see a bust like we did at the beginning of 2000.  With low cost hardware, better software and access to high speed internet, the VR industry will only grow from here…. Slowly.

Image from gizmodo

Not too impressed, but I cannot wait to see an actual game like in the Gear VR.  The quality could be a bit better, but still pretty good.  The NBA All Star video should be out in a few weeks so we are on standby.  REALLY looking forward to it though.

There are two videos, both just over a minute. The 3 point video has 3 guys, two collecting and feeding the ball, one doing 3 point shot exercises.  Hitting .. every… one.. by the way.  You are standing at the top of the key watching.

The second video is similar but instead you are shown some basics of the game.

The videos do a good job of putting you in the scene, but somehow it still felt like it could have better depth punch.  Could be I was looking at this after testing out 2 other CG Gear VR titles and I was expecting too much. Also, lighting seemed pretty general and its just some guys shooting, a game, with more action will be better.

I reasoned that what we need to is a ‘pod’ where the floor base was made of small wheels and the waist was restricted by a harness.  This will allow the subject to simulate walking and turning thus allow more of the bodies haptic feedback loops to agree with what was being seen in the HMD.  Although VOR gain adaption would still be needed, the adaption time would be greatly reduced, and the in VR sustainability time extended, as I discovered with my work on the chair and allowing free walking. However, I never could build such a device because of cost.   But now, someone has figured it out.

Originally starting out as a kickstarter project, the ‘Omni‘ is an improved version of what I designed back in the 90’s.  Instead of using rollers, a grooved curved plastic bowl is used with sensors. (older version uses an xbox sensor)  These sensors can translate movement and direction to the game. Paired with the new lightweight Oculus, the user walks normally and turns normally.  Admittedly it does take a little bit to adjust to your feet slipping instead of the force feedback with normal translational movement, but the connection between body mechanics and spatial perception is synced enough that adaption happens quickly. The rig itself is $499, quite affordable for most serious gamers. What is exciting about this we might be seeing the beginning of the next level of maturity around VR that was stunted in the 90’s. With inexpensive but capable HMDs and now inexpensive VR motion platforms, we can make that next leap in immersive gaming.

//www.youtube.com/watch?v=iUkjaNQlZO8

______________

Excerpt for my Master Thesis: Spatial Perception in Virtual Environments:
Visual Cognition Gain with Head Mounted Displays (images removed)

VOR

Despite the commonality of simulator sickness, little is known about it.  The most common source of simulator sickness is Vestibulo-ocular Reflex (VOR); The vestibular apparatus is a small structure that exists in the bony labyrinth of the inner ear, whose function is to sense and signal movements of the head. This function is extremely important because it contributes to the coordination of motor responses for the body.  VOR is the fundamental eye-movement reflex that functions to keep images stabilized on the retina during movement of the head, thus helping to perform a very basic but important function, to allow sight during movement. The disagreement between what is seen by the body and what the vestibular apparatus senses can cause confusion, providing disagreeing signals to the brain and thus inducing sickness.  VOR can adapt its signal to the brain when the environment demands it, but not everyone has the same VOR adaptation ability.  This inability to resolve vestibular and ocular discrepancy is a major contribution to simulator sickness.  The greater lag time between screen position updates and vestibular sensation can accelerate simulator sickness even more.

So I just got back from the ESRI User Conference 2013 in San Diego and wow, what a trip.

Founded by Jack Dangermond, ESRI (Environmental Systems Research Institute) is an industry leader and international supplier of Geographic Information System (GIS) software and geodatabase management applications. The company has an installed base of more than one million users in more than 350,000 organizations, including most US federal agencies and national mapping agencies, all 50 US state health departments, transportation agencies, forestry companies, utilities, state and local government, schools and universities, NGOs, and commercial business.

The user conference brought 15,000+ ESRI users from over 130 countries. Keynote presentations included Advisor to the Prime Minister of India on Public Information, Infrastructure, and Innovations Sam Pitroda and an inspiring discussion with pop culture icon Will.i.am. For the record I am a B.E.P. fan but this interview was something else.

This was my first time to attend the ESRI User Conference and I was amazed at the number of sessions, expo size (entire convention center floor), attendance, and diversity of the show. As an attendee pointed out, “GIS is everything” and it seems everything was represented at the event. Among what I saw on the trade show floor were a variety of mapping applications, LIDAR services (remote sensing technology that measures distance by illuminating a target with a laser and analyzing the reflected light) and drone data collection devices.

I am not a geo-specialist but I did recognize a few things I am familiar with. Among those was:

City Engine – a procedural modeling tool for rapidly creating large urban environments

Geoloqi – a platform for building location aware applications with geotriggers and realtime location capabilities.

Metaio – an Augmented Reality platform provider

One ESRI invited exhibitor was building company IDEAbuilder. IDEAbuilder uses geospatial data for a variety of uses in the building process and were showcasing a new application the company tech team has built that combines interactive 3D and digital fabrication with some ESRI tools and services. This was a great demonstration of how 2D/3D building data can connect with geospatial data to create highly intelligent interactive scenes.

The application that was shown was the virtual reality version of their Performance Shell Configurator or PSC. The PSC application is a green building tool for their builder partners that provides the builder the ability to go into an interactive 3D scene and snap together the shell of a structure from a library of digital fabrication ready components that can be robotically manufactured. These metadata rich components provide their partners with information useful to the building process. I posted several months back the web3D and Kinect version of this app that got covered on Wired.com (thanks Bruce Sterling!)

Some features of our Oculus Rift virtual reality version include Razer Hydra hand controller support, the ability to import a blueprint image to use as a guide to build on top of, a live top down map so you can see where you are on the blueprint, a total structure cost calculator, elevated cherry picker view, a virtual ‘AR’ view where staring at a panel piece displayed part information including cost, and free navigation of a City Engine city that acts as the 3D backdrop to the PSC building space. There will be an official post with more information about the PSC app on the IDEAbuilder site so stay tuned over there.

The application was a huge hit and really helped visitors experience new ways of interacting with data. While most people were unaware of the Oculus Rift or have never had a virtual reality experience before, there were those that were more than ready to get started like John Bormann. In addition to the PSC app, they also shared an Xbox 360 controller controlled architectural visualization Oculus Rift scene created by real life architect, web3D supporter, and IDEAbuilder friend Jon Brouchoud.

In closing, I would like to thank the IDEAbuilder team for this cool virtual reality experience, everyone I met at the event that helped me understand more about GIS, and the people at ESRI for the opportunity and everything they did to make it a super ‘geospecial’ time. See you next year!