Final Project
For Final Project I use the way to overlay a virtual glass
over a real glass due to the technical limitation that AR cannot recognize
transparent object or reflective surface at the moment. Therefore, I had to use
the alternative way.
After I made the technology works in Unity, I thought of the
findings I got from the user interview.
Even though the AR level indicator functionality is cool and
useful, it doesn’t get enough motivation to have them download the App.
Therefore, I decided to make AR as a feature of a cocktail
recipe App instead of a standalone App.
I designed the logo with Illustrator, the screens with
Sketch, and prototyped with Framer Studio.
During my demo to the NYC Media Lab and The New York Times,
I received high compliments.
-------------------------------------------------------------------------------------------------------
Week 9 Cocktail Making App MVP
From my user interview session, Renee introduced me a set of glass for cocktail making which she bought from MOMA. It is very inspiring and can be very helpful to refer when designing and making the AR App. The 4 glasses mean cocktails you can make with the 4 different base. On each cup, each side of the 4 sides indicate a recipe for a cocktail, and marks the line for each ingredient. Following is a video Renee introducing this glass set.
Also, I contacted the one who posted the AR cocktail mixing video on Youtube that Mark found and waiting for him to reply now.
Furthermore, to make my idea more clear, I made a prototype of the key screens.
For Final Project I use the way to overlay a virtual glass
over a real glass due to the technical limitation that AR cannot recognize
transparent object or reflective surface at the moment. Therefore, I had to use
the alternative way.
After I made the technology works in Unity, I thought of the
findings I got from the user interview.
Even though the AR level indicator functionality is cool and
useful, it doesn’t get enough motivation to have them download the App.
Therefore, I decided to make AR as a feature of a cocktail
recipe App instead of a standalone App.
I designed the logo with Illustrator, the screens with
Sketch, and prototyped with Framer Studio.
During my demo to the NYC Media Lab and The New York Times,
I received high compliments.
-------------------------------------------------------------------------------------------------------
From my user interview session, Renee introduced me a set of glass for cocktail making which she bought from MOMA. It is very inspiring and can be very helpful to refer when designing and making the AR App. The 4 glasses mean cocktails you can make with the 4 different base. On each cup, each side of the 4 sides indicate a recipe for a cocktail, and marks the line for each ingredient. Following is a video Renee introducing this glass set.
Also, I tried to implement it with Vuforia. I was thinking about using object target. However, the scanning provided by Vuforia only supports Android and some documents say it is only working with opaque items. Making sure the glass is transparent that users can see through is an important point mentioned in the user interview. I also tried sticking an image target on the surface of the glass while the overlay is not very steady and will be hard to align when pouring alcohol. Therefore, I wish to hear more technical advice from Mark and the class.
Upon from the MVP, I would want to add in more recipes, searching and suggestion functionality, detection of different shapes of glass for further development.
-------------------------------------------------------------------------------------------------------
Week 8 User Interview about AR App for Cocktail Making
For my final project, I am thinking about building an AR App for making cocktails at home. When position your camera in front of the glass, it shows you lines to indicate how much each ingredient should be added.
User 1: Nelson Oliveira
User 1: Nelson Oliveira
Nelson works as a bartender and at a cafe before. Besides work, he has also tried to make cocktails and coffee at home. He usually eyes the proportion and tries as he makes the drink. He pointed out that the ice cubes on bottom will be a challenge to mark the line for liquid. He feels it would be more useful for iced coffee making than cocktails.
User 2: Chris Yuming Shi
Chris Shi loves making cocktail and has a small bar at home. He introduced me a product he saw before. It is a digital scale with voice instruction for cocktail making. However the product failed because professional bartenders don't need this tool to help with the recipe, and for beginners the price of 100 USD is too expensive to give it a try.
Also, Chris raised problems that may arise using the App, like how to measure the volume of cup, and the volume of the ice cubes. Different cocktails require different amounts of ice, and sometimes you shake it and sometimes you stir it and it affects how much ice melts. Additional feature he suggested is to allow users to choose a recipe based on preferred taste (strong or sweet) and existing materials (what can I make with vodka and orange at home?).
User 3: Tarun Shyam
Tarun is a entrepreneur who has two agencies in Dubai. He also loves cocktail and tried to make cocktails at home. He suggested that some people may like variations based on the recipe, like some people want it to be stronger. Therefore, options to customize based on the selected recipe would be useful.
User 4: Renee Wang
Renee loves cocktail and loves DIY things at home. However, she never tried to make cocktails at home. The biggest challenge for her is the recipe and proportion. Therefore, the App could be helpful. She raised a problem that users need to use transparent glass instead of mug to see through.
Renee loves cocktail and loves DIY things at home. However, she never tried to make cocktails at home. The biggest challenge for her is the recipe and proportion. Therefore, the App could be helpful. She raised a problem that users need to use transparent glass instead of mug to see through.
User 5: Till Daldrup
Till tried to make some cocktails at home before but mostly just guess the proportion. He said feature to allow users to make it lighter or stronger than the recipe would be useful.
Week 7 Spring Break
Week 6 Refined Idea about Final Project and Interview Questions about Target Audience
I pitched the department store lighting correction idea last week while I am not very excited to do it for my final project. Therefore, I started to brainstorm again and came up 10 ideas in between the two weeks which I listed in the part of Week5 blog.
After discussing with Mark about the feasibility and usability of each idea, I narrowed it down to one idea which I feel excited to further explore: AR in plastic surgery.
Then I started to do my research.
Research
In 2016 ILLUSIO built this App for patients and surgeons to preview the outcome of breast augmentation surgery.
In August 2017, Koichi Ueda, MD, PhD, Daisuke Mitsuno, MD, and colleagues of Osaka Medical College used AR in plastic surgery and reconstructive surgery which enables them to create 3D simulations of the desired results of facial reconstructive procedures and project them over the patient's face during surgery.
Other related findings include:
In 2009 Vectra 3D uses 3D imaging to generate simulation for breast augmentation, chin augmentation, rhinoplasties and liposuction.
In 2017 Swiss startup Kapanu in 2017 uses AR for dentistry
In 2018 Imperial College London uses Microsoft HoloLens headsets to overlay CT scan images while operating on patients undergoing reconstructive lower limb surgery to show tissues and blood vessels.
In 2016 Fushima and Kobayashi suggested a mixed reality-based system using a dental cast model and a 3D maxillofacial mesh model.
Final Project Idea
Therefore, the refined idea for final project will be an AR App for one specific type of plastic surgery, like nose bridge augmentation, or double eyelid surgery. After hearing the feedback from the class, I will start to talk to surgeons and technologists about which specific surgery will be more feasible for the final project scope.
Target Audience
The target audience for this App will be plastic surgeons and patients who need better visualization of the surgery outcome to facilitate communication.
Interview Questions
Questions for surgeons:
1. Mechanism of the surgery and how different factors affect the outcome
2. How much will the uncontrollable factors (like rejection to to implantation) affect the surgery outcome?
3. Existing method for communication and problems with the existing solution
4. Usability of this App
Questions for patients:
1. The biggest concern with existing solution
2. Usability of this App
-----------------------------------------------------------------------------------------------------------------------
Week5 Wearables & Department Proposal && Final Project Thoughts
Wearables
I planned to do a superhero costume, or pirate costume with hat, eye patch and sword, or santa costume with white beard, hat, and present. However, resources are so limited. In the end I can only find this. However, it works but super unstable. I need to get really close to the camera and it still shakes so much.
Department Store
Regarding the project that uses AR to improve department store shopping experience, I wish to propose an App or a add-on feature to existing App that can help users view how the clothes look like in different lighting environment.
Many department stores have different lightings with some warmer some darker etc. This affects how the same outfit looks.
The Dress is an example how lighting affects our recognition of color.
Kristin Chirico from Buzzfeed has did an experiment how the same person looks like in 20 different dressing rooms. Read More.
MemoMi produced a high-tech mirror for make-up which can help you see how the makeup looks in Studio, Daylight, or Night. Read More.
Therefore, an App that can help customers see the real color of the outfit, and how it looks like in different lighting environment such as natural outdoor sunlight, indoor warm light, indoor cold light, at night etc can be helpful. I made a prototype to show a sample idea.
Final Project Thoughts
1. AR in plastic surgery
Now doctors usually use photoshop or just description to communicate with patients about how it will look like after the surgery. Photoshop gives a visual but usually it is only front and profile. With AR patients can have a 3D view and communicate with doctors better such as how wide they want the double eyelids to be, how high they want the nose bridge to be raised etc.
2. AR in Chinese medication
In Chinese medication there is something called acupoint which are key points on your body where you press acupuncture to help your body channels to flow more smoothly. With AR, you can know where are the acupoints and enable you to have home treatment by yourself.
3. AR in film industry
Currently many film companies need to build construction or shoot in green screen and use CG to build up the scene. The first way costs too much time and money, and the second way challenges the actors more acting based on imagination. Therefore, if companies can use AR to build virtual scene in the green screen space, it will help actors to perform better.
4. AR translation for foreign language
Use AR to overlay translation on what you see. Google has this already and it is really helpful for people to travel to a foreign country. I was thinking, if it is possible to integrate real-time translation of spoken language as well (Though Google has it too).
5. AR translation for people who use sign language
For people who cannot hear, AR helps translate the spoken language into text. Also, for abled people who are communicating with people who use sign language, AR helps translate sign language into text/sound.
6. AR calorie indicator
With AI, the App recognize what the dish is and calculate the portion size based on the image size and distance between it and the camera. Hence, calculate the calorie and show it on the screen.
7. AR makeup tutorial
The App shows virtual lines like which area should put on shadow and which part should put on higher, and how the shape of the eyes shadow should be, to help girls learn makeup more easily.
8. AR violin tutor
Violin is especially difficult when it comes to that it doesnt have frets and it is hard to get the right pitch. With AR glasses, it shows the frets and tuner about you are lower or higher than the right pitch.
9. AR workout assistant
When you are working out, showing which muscle are you working on and if you are doing correct to use the muscle that you wish to exercise.
10. Easier AR building tool like Wix to web programming
After learning AR for a while, I feel it is not that hard to build a basic AR like placing objects. Aurasma and Layar are not very user-friendly. If we can build something that lowers the requirement to build AR and allow more people to use AR, it would be helpful.
------------------------------------------------------------------------------------------------------------------
Week 4 AR with Vuforia
I first tried to build a piano with floating notes coming out like the following.
However, I tried to debug this but it always show only black screen when clicking on the play button to test.
Therefore, I thought about doing something else and came up with the Merlion spraying water which is a symbol of Singapore.
I managed to do it but I felt it is too similar to what is covered in the tutorial.
Thus I had this idea to build a model overlay on the floor plan map to give people a closer-to-life feeling of how the room will look like.
-------------------------------------------------------------------------------------------------------------------------
Week3 Intervention with Layar
I got my inspiration from the movie KingKong and want to create something connecting it with the Empire State Building in AR.
When KingKong fell from the Empire State Building, he actually survived due to certain superpower. Human agreed to let KingKong lives in the world like an average people, study and work.
Now in 2018, Darrow has passed away and King Kong is still studying trying to get a Bachelor degree.
Every time when King Kong feels stressful, he looks from the huge window of Bobst, and the Empire State Building reminds him of the good time with Darrow.
-------------------------------------------------------------------------------------------------------------------------
Week2 AR Experience with Aurasma
AR Humanized iPhone by Nicole Zhong
I imagined if my phone has a personality, what will it be? Therefore, I created this video with After Effects and overlay it upon the phone model.
-------------------------------------------------------------------------------------------------------------------------
Week1 3D Modelling with Maya
In 2012 summer, I visited Phuket Island in Thailand with my friends for holiday. The beach was awesome in the sunset and all the people were having so much fun.
No comments:
Post a Comment