Read Mixing movement and machine by Maya Man
movement and machine and minds merges
it shows a way of narrative other than speaking + dancing
Read Humans of AI by Philipp Schmitt
“Computer vision, or artificial intelligence in general, certainly isn’t magic. Neither is it merely data. Each prediction by a computer vision algorithm is an encounter with the work of collective memory.”
Pick one of the models above (PoseNet, HandPose, UNET, BodyPix, CoCoSSD) and following the examples and ml5.js documentation experiment with controlling elements of a p5.js sketch (color, geometry, sound, text) with the output of the model. (You may also choose a ml5.js model not covered here if you like!)
- https://editor.p5js.org/yining/sketches/W5MU2Xc02 (nose drawing)
- https://editor.p5js.org/yining/sketches/NhSshC1cq (pose + cat)
- https://editor.p5js.org/yining/sketches/jSnEyDrPw (Bodypix color )
- https://editor.p5js.org/yining/sketches/pVNmUDn-Z (Object detection)
- https://editor.p5js.org/yining/sketches/Tb4V-fXaz (trailing)
- https://editor.p5js.org/yining/sketches/_qolqm9Bo (Drawing)
- https://editor.p5js.org/yining/sketches/I_qrc4y6J (wrist growing rect)
Considering the Model and Data Biography reflect on the the following questions:
- What questions do you still have about the model and the associated data? Are there elements you would propose including in the biography?
- Who get to decide what information are allowed and interpreted in the dataset?
- To what extent are the information in datasets real?
- Can machine learning models be represented physically?
- I would like to add “possible ways of implementations” into the biography
- How does understanding the provenance of the model and its data inform your creative process?
- I think it reveals how the machine learning models are created from datasets and informations
- Although ml5 PoseNet and UNET are both created from the COCO dataset, one focuses on image classification, object detection, and image segmentation for a person’s skeleton while another focuses on the pixels region of the body.
- Inspired by the readings and the informations about the models, I found that I am interested in learning about how can we tweak information and pixels of an image? And combine body movement with machine learning?
Document your response to the Data and Model biography as well as your p5.js sketch in a blog post and add a link to the post / your p5 sketch on the Assignment 3 Wiki. In your blog post, include visual documentation such as a recorded screen capture / video / GIFs of your sketch.
patrick star https://editor.p5js.org/tl3417/sketches/Th1wPP4uD
patrick draws https://editor.p5js.org/tl3417/sketches/Th1wPP4uD
booom https://editor.p5js.org/tl3417/sketches/69atr_Iv6
Screen Recording 2023-10-05 at 15.50.31.mov
moving triangle https://editor.p5js.org/tl3417/sketches/Bf2D01tf0
Screen Recording 2023-10-05 at 15.49.19.mov
some accidents:
