With the arrival of post-PC era, more and more smart phones, tablets and other portable devices are on the way. Many companies nowadays are devoted to developing wearable devices, such as Google, SONY, Nike and Qualcomm. Without question, smart watches are the most popular one among these. Although there are many smart watches from different companies on the marketplace, not all of them earn customers’ favor. Launched in September 2013, Samsung Galaxy Gear smart watch is poorly reviewed by industry experts and end users. It is rumored that the Galaxy Gear had a return rate as high as 30% in the US, as a proof that many customers are not satisfied with this product. According to Ian Fogg, an analyst of IHS, he concluded: "The device exhibits multiple shortcomings, including a high price tag, a short battery life, and its limited compatibility.” Through this case, we can see that smart watch in today is still an immature product, and has a lot to go before it gets big and red.
Picture 1: "It appears that Samsung’s Galaxy Gear smart watch is a prototype masquerading as a commercial product." said Ian Fogg, director for mobile and telecoms at IHS.
In general, there are two kinds of smart watches in terms of user interface (UI) for customers to choose on the market: one is equipped with touch panel and the other one is controlled by pushers and some are able to take pictures while some have voice control function. As a certification expert, Allion Labs, Inc. (Allion) keeps up with the times and shows concern about the trend and development of smart watch on the market. In order to know the real performance and make sure the UI is user-friendly, we recently choose three different products from notable companies to evaluate their performance by five power users. This test report is mainly focusing on the appearance, UI and functionality of the product.
Pic 2: DUTs from left to the right:
SONY Smart Watch 2, Martian Passport Watch, Pebble Smart Watch
The watches we purchased this time are released from Q2 to Q3 2013. As in pic 2, from left to the right are SONY Smart Watch 2, Martian Passport Watch and Pebble Smart Watch. Unlike ordinary portable devices usually with similar appearances, smart watch individually possesses different styles. SONY Smart Watch 2 adopts touch interaction, with a 1.6” 220 × 176 pixel LCD monitor. Martian Passport Watch combines traditional watch dial with a 96 × 16 pixel OLED monitor which only displays words. On the left side of the dial are two pushers and on the right side is a screw down crown for adjusting watch hands. Pebble Smart Watch has a 1.26” 144 × 168 pixel e-paper monitor. There are four pushers in total on both sides of the dial and just like Martian, it doesn’t have touch function.
Chart 1: Spec details of three DUTs
For the accuracy of this report, the evaluation of this test report can be divided into three categories: appearance, UI and functionality, according to the themes. Five power users would rate each test items, score 10 as the most satisfied and score 0 as the most unsatisfied. If the product cannot support certain function or compatibility failure happens, then it would be rated as 0 point.
Chart 2: Appearance and UI point chart
- Appearance
As in chart 2, SONY Smart Watch 2 with metallic-luster watch dial touch panel performs better in “Comfort of wearing” and “Sense of design” than Martian Passport Watch with OLED display and Pebble Smart Watch with only e-paper display. However, since SONY uses butterfly style buckle, which means it’s hard to adjust tightness of the belt, Pebble wins in “Convenience of wearing”.
- UI
SONY beats everyone in three test items: “Interface fluency”, “Font size” and “Font identification”. It indicates that users are fond of this 1.6” monitor that can display six APP icons at the same time. However, SONY’s home button sometimes does not respond. Besides, Pebble’s design of four pushers on both sides of the dial which are hard to press is not convenient for users, so it gets the lowest points.
Chart 3: Functionality point chart
- Functionality
Smart watches on the marketplace are mostly an extension of smart phones, so smart watches need to pair and interconnect with smart phones or tablets first, and then the smart watch will be able to be controlled by APPs. Therefore, we individually separate test items for “Satisfaction Survey on APP on Smart Phone” from functionality test, so that we can validate this feature precisely. Since SONY is the only one that is able to pair with smart phones through Bluetooth with its built-in NFC, with just a click which is faster than anyone else, it gets itself the highest points in “Usability of pairing process with smart phone” and also gets itself in the first position in test items “Operation interface” and “Usability of APP”. In “Satisfaction Survey on Basic Function” test item, SONY performs better in “SMS notification” and “Mail (Gmail) notification” but loses to Martian in “Incoming call notification”. Because Martian is the only product that can perform voice dialing and is able to pick up incoming call and conduct conversation through its built-in noise-canceling microphone and speaker. But Martian’s voice dialing function is not accurate and has high failure rate since the language of voice control is not supported in the area users reside in.
Currently smart watches play a role that mainly assists with smart phones through Bluetooth technologies to inform users when smart phones receive new notifications or incoming calls. Therefore the stability of interoperability (IOP) between smart watches and smart phones is relatively important. In this test report we select eight different smart phones including six Android phones and two iOS phones to run the interoperability test with these three smart watches. The test coverage includes major functions such as SMS notification, incoming call notification and social network notification. It is clear in chart 4 that three smart watches all have IOP problems with corresponding smart phones. For example: SONY cannot receive Mail and Facebook notifications from LG Nexus 4; SONY XPERIA Z receives Facebook notification but Martian does not.
According to Allion experts’ points of view, the results indicate that companies need a multi-angle IOP test plan to make sure smart watches can work well with smart phones and to level up product quality and user experience. Moreover, Allion’s Bluetooth Qualification Expert (BQE) finds out that although Martian Passport Watch claims that it adopts Bluetooth 4.0 Low Energy technology, but actually there is no available profile that can perform Low energy technology based on the test result. Despite Pebble lays claim that its smart watch is equipped with Bluetooth 4.0 technology, there is a discrepancy because its spec is Bluetooth 2.1+EDR, which differs from Pebble’s claim on its website. Thus, developers need professional Bluetooth certification test and consultation service to ensure that products follow Bluetooth spec regulations and are able to work correctly with other devices.
Chart 4: Interoperability test with smart phones
Overall Review:
- SONY Smart Watch 2
It is a stylish product with high scalability. Generally speaking, UI performance is good but if the resolution can be enhanced to attract more potential customers’ attention. Every APPs on smart phones need to be installed one by one which makes it more complicated and time-consuming for first-time users. Monitor’s sleep mode doesn’t adopt smooth fade-out design and this would confuse users, causing users to doubt whether their product is facing a crash or monitor encounters problems. After testing APPs of some famous social networks, we find out that users cannot set all posts on Twitter as read and SONY might lose Facebook notifications or have notifications delay. Besides the malfunctions, we suggest that it would be more user-friendly if users can adjust the location of APP on the monitor freely and the monitor can sense light and adjust lightness automatically. Also, It would be better to have voice control function too.
- Martian Passport Watch
APP on smart phones is very easy to use, not complicated. But watch dial is too cumbersome and not comfortable to wear. Besides, its Bluetooth connection failure rate is too high, some functions of APP are not able to work properly; for example, voice dial identification accuracy is low and camera that turned on by watch is unable to do autofocus. Watch dial desktop is too dull. It is better to enlarge its monitor and lighten its thickness and weight.
- Pebble Smart Watch
It is inconvenient to download data to smart phone first and then dump to smart watch, which is not easy at all for users to install APP. Plus, 11MB-size APP is relatively big. APP interface is too simple and there are too many word instructions in setting page, making users hard to read. In installation mode, there are no explicit instructions telling users which pushers to press on the right side. Not supporting multi languages is also one of the biggest shortcomings. When the connection is terminated, APP on smart phone still keeps searching devices, causing high power consumption for smart phones.
Suggestion from Power Users:
The coordination between software and hardware for developing a perfect product is significant because customers are expecting a unique, creative, and reliable product. We are looking forward to seeing SONY and Martian having more standby theme choices for smart watch monitor. For Pebble that uses e-paper monitor, it would be nicer if LED lights can turn on automatically, so users can use it more convenient in dark environments. Developers should simplify all processes when installing APPs on smart phones and arrange simple pictures to instruct users step by step instead of complicated interfaces with too many words. In consideration of waterproof design, SONY and Martian all set a lid to cover charge port. But this causes the lid to be stuck on the screw down crown and is easy to break when Martian’s smart watch is charging. All those details mentioned above would influence customer’s user experience and satisfaction. In addition, we also suggest smart watch should offer navigation function and add healthcare service to fully bring out the advantage of Bluetooth 4.0 Low Energy technology.
Pic 3: Pebble Smart Watch’s exposed metallic charge port is convenient to use but it also has high possibilities to cause short circuit.
Currently the reason why customers are still thinking whether to purchase or not is mainly due to smart watch’s high price, inadequate design, lack of creative innovation and short battery life. Moreover, there is only a very limited amount of APPs for smart watch on marketplace, which would deteriorate potential customers’ purchasing intention. In this test, there is no denying that compatibility problems between smart watches and smart phones are ubiquitous. Therefore, developers need to take interconnect ability of smart watches with other devices into first consideration in order to develop more advanced functions. Allion’s BQE forecasts that if smart watches want to go on a revolutionary innovation, it may be like, for example: 1. offering healthcare & fitness function, 2. combining with fitness sensor and Bluetooth 4.0 Low Energy technology and based on the advantage of wrist and smart watch being closely associated with each other to provide more extensive services than smart phones. As a professional test lab, Allion can conduct one-stop certification test of touch function and Bluetooth technologies. We also have a variety of the latest portable devices collected quarterly for interoperability test with clients’ products. Moreover, we also provide Competitive Analysis for clients’ products and competitors’ and real-time pinpoint the possible root causes, helping clients solve the product development dilemmas.
No comments:
Post a Comment