Week 7

Posted on by Ankit Patel

Hi everyone. Welcome to week 7 of our project. This week we started working on different mechanics that fits into camera, custom markers, Cozmo’s LCD face display.

SDK Update (v0.7.0):

Midweek we got an update of Cozmo sdk. In the new SDK, they have fixed couple of issues based on our past requests on forums. Here are the few major updates that were fixed in new SDK.

  • In the SayText class, the use of pitch and unprocessed voice has been fixed.
  • In faces.py and objects.py, new events EvtFaceAppeared, EvtFaceDisappeared, EvtObjectAppearedand EvtObjectDisappeared fire when a face or object appears or disappears. Also, new methods visible_object_count and visible_face_count in world.py have been added to count the number of currently visible objects and faces.
  • Other additions:
    • The DriveStraight action has been added to robot.py and is used by examples includingsimple_script.py and drive_square.py.
    • Method set_robot_volume has been added to robot.py.
    • In robot.py, the lowest and highest lift settings are exposed with MIN_LIFT_HEIGHT_MM andMAX_LIFT_HEIGHT_MM.
    • The tap_intensity field has been added to EvtObjectTapped in object.py.
    • Added robot.abort_all_actions.
    • Fixed an issue where action.abort() was not aborting many action types.
    • The BlockTapFilter can now be disabled in world.py.
    • A Tk error has been fixed for machines with no valid DISPLAY environment variable.

Consumer Mobile App Release:

They have also released official consumer version of the app both on Appstore and Playstore. Since Cozmo is released on Oct 15, now we have access to consumer app and we can still develop custom games and experiences by enabling SDK mode (Settings > SDK mode) in the mobile app.

Process:

Initially we brainstormed a lot of ideas that uses Camera, Custom markers, LCD face display and Drawing. Each programmer worked on developing one prototype. Links of the videos are posted.

So the prompt for the week was “Camera, Custom Markers”  We focused on developing prototypes based on our brainstorming session

Development:

Custom Markers

Since we had access to Custom marker images in new sdk, we thought to experiment with that to use more than 3 objects( cubes) with Cozmo which might be useful for creating complex games in future as well. In first two days of testing with Custom Markers, we found that those markers are not reliable to identify new objects in Cozmo’s world. Cozmo was able to detect custom markers, but it detects as a different object every time he sees it. Also this behavior is same even if we use single marker. We also posted these issues on forums.

Since there do seem to be a number of issues with the custom markers currently, they have also demoted “Custom Markers” to be “private” for now for the first public release of the SDK, and they suggested people hold off on using them until they can make that system more robust.

Cozmotography

Another idea we worked on this week was to use Cozmo as a photographer and that’s why we named it Cozmotography. We tried OpenCV tools to analyze images from Cozmo’s camera and also used Google Cloud Vision API to search for keywords from image. In current experience, Cozmo asks user to put some object in front of him and searches through web for related keywords using Google Cloud Vision API. For instance, if we put glasses in front of Cozmo, he can investigate and says words like “Eyewear”, “Glasses” etc. We will continue working on this in next week to get more accurate information ( Text, Emotion etc.) and may also integrate Snap Chat API to post images from his Camera.

Learning Mechanics (using LCD face display)

We received interesting comments for using Cozmo as a learning tool for Kids from lot of people who visited our project room in past few days. We explored a lot of educational ideas in the beginning, here’s an example of one of the games we developed –

A. Cozmogram

An educational game where the guests are playing 3-letter anagram with Cozmo so we named it “Cozmogram”.  Idea here is to display 3-letter anagram word on his LCD face display and each letter is mapped with 3 cubes in same order (left to right). Goal of the player is to move the cube in order to move the corresponding letter on his screen to make write word. Cozmo will react based on correct/incorrect answer. Currently there is no time limit but in more complex version of this game, we may include timer and also more than 3-letter anagram using Makey-Makey

https://www.youtube.com/watch?v=bN0_qJbxPIg

 

 

Thanks for checking in! Do drop by next week for more updates!