Posts

BIG NEWS

I got through a lot of really exciting stuff this week! New microcontroller achievements, parts ordered, things coded, etc! The big advancements come as part of the project I’m doing with Mila and Thomas to refurbish the submarine game. See below for some exciting advancements not just in the project but also in my overall thesis explorations.

Microcontroller Experiments

So for the past week or so I’ve been following tutorials online and learning about Microcontrollers, circuits, etc. I started my journey a while back when I was tinkering with the Adafruit Bluefruit board to do some simple coding and interactions. I’ve since then moved on to Arduino, and more specifically Arduino UNO and Leonardo. Both boards serve a similar purpose, but I’ll get to why I’ve tinkered with both in a bit.

Ultimately, my goal this week was to get an LED turned on, a button to work, and a keystroke to register through the board. I’ve successfully met all goals through 3 mini-projects that I did this week. The first one was the button press experiment. This one was fairly easy and required just a bit of breadboard setup and circuitry; I learned a ton about the basics of circuits and the fundamental principles of circuit-building. Next was getting a push button to trigger an LED.

This project was a bit more challenging and required some additional wiring and coding to get a simple system up and running.

Once this challenge was tackled, I started evaluating how I can get the microcontrollers to communicate inputs into keyboard presses. This resulted in a smashing success with a microcontroller button press initiating the notepad app by typing and searching for it in the Windows taskbar!

So now that I’ve figured out how to connect the microcontroller and rig simple buttons and controllers, I’m moving on to connect and develop the full console that will be used for the submarine game. This big console will be made from upcycled Styrofoam inserts that are spray-painted and decorated to be themed to a submarine environment. The buttons will then be inserted and built into the console using foam board and clever cut-outs to make the consoles feel heavy and immersive. I’ve now ordered the joystick and buttons that will be featured on the console, and will begin assembling everything when they arrive next week!

Introduction

So this semester I’m starting off by working with Thomas and Mila to refurbish and improve their existing 2-player VR submarine game. The game consists of an Oculus Quest 2 that shows the view from a submarine’s periscope while player 2 navigates based on what is shown through the Oculus. Our goals for this new version are to improve environment assets, develop a new and more interesting level, improve existing mechanics, and create a themed control system for the person navigating the sub. My role in this project will be to develop and build the microcontroller-based console and connect the Arduino inputs to the computer for use within Unreal.

Progress Update

We’ve finalized our controller layout ( for some reason the image I’m trying to upload for that keeps turning up blank) and I’m now moving on to experimentation using this neat little kit I got at Microcenter. The kit features various buttons and LEDs to help develop and prototype simple input methods. I’ll be making some simple layouts with buttons and LEDs to then work with Thomas on connecting the inputs to Unreal.

Thesis Committee

So my Thesis committee is comprised of Dr. Matthew Lewis, Shadrick Addy, and Alex Oliszewski. Alex is on sabbatical this semester so he won’t be too active in my meetings until next year, however I do plan on updating him during my second year review. The first meeting was with Matt and Shadrick and it went really well. We sat and discussed my thesis packet ( which I will upload here: Thesis Committee Request) as well as some next steps I can take to continue refining my topic and future thesis document.

Notes During the Meeting

• How can I phrase my research question and thesis approach as something that addresses a specific gap and also benefits other facets like accessibility.
  • They also suggested I pull away from a deep focus into accessibility.
  • Instead focus on a couple elements like audio impairment, motor impairment, and wheelchair users.
• Defining my concept of “interaction” – passive vs. active interaction.
• What exact types of “unique input methods” am I talking about.
  • Defining my unique interactions as physical versions of normal VR interactions like grab, touch, press, pull, etc.
• Who is my research aimed for? I answered this question by mentioning my research is aimed towards innovators and designers within the themed entertainment that want to think about and explore new methods of combining VR with themed spaces & attractions.
• Both Shadrick and Matt suggested I read Sarah Caudill’s paper to gain some clarity on how an internship can influence my future thesis.
• They suggested I begin transitioning my big packet into a working draft of my future thesis, start filling in areas here and there as I conduct my research.
  • Perhaps include a 1-page walkthrough of my experience for those that cannot actually go through it.
• Approaching my research through a lego-based model by starting with a base and exploring additional levels of research but always having something built and ready that works on some level.

Next Steps

I’ve now decided to meet with Matt (my primary advisor) every two weeks to check in on progress and get some help with anything I may be struggling with at the time. These meetings will also help me continue to formulate my thesis writing and research direction.

Introduction

So the semester has started again and I’m off to a great start! Spring 2023 classes include Scenic Design, Grad Studio, and a UI/UX course with Jeremy. I’m looking forward to solidifying some microcontroller experiments, and beginning some scenery exploration for my final thesis. Each class will contribute to a unique element of my thesis research:

1. Scenic Design will help me develop and think about themed elements in the context of my installation.
2. UI/UX Design Survey will help me evaluate and understand how to incorporate accessible and inclusive UI elements in my experience.
3. Grad Design Studio will feature some group projects and personal explorations of microcontroller systems for Unity/Unreal.

I’m really looking forward to this semester and the work has already begun!

Research Planning

So the biggest project this semester will likely be the microcontroller projects in Grad Design Studio. I’ve already been doing lots of research and learning about the topic. I’ve gotten some practice with the Makey Makey microcontroller (MC) as well as the Bluefruit board. I plan on learning and using Arduino for my final thesis experience, so creating some tests/research for that are at the top of my list.

The current project in mind is to work with Thomas E. and Mila G. to improve and expand upon their original game idea of a collaborative VR submarine game. I will work to combine buttons and switches to create an exciting and un to use interaction system. The element will be themed to the game and work as my first exploration into actually building systems like this.

I will also be working with Dr. Lewis to help our team develop this system and answer any questions we might have during the process.

Committee Updates

The three committee members that were requested were Dr. Matthew Lewis, Shadrick Addy, and Alex Oliszewski. All three have giving me verbal confirmations of joining my committee, but Dr. Lewis has officially joined as my primary advisor. I will be meeting with my committee towards the end of the month and I plan on updating them on the projects I’m planning for this semester as well as my plans for the summer.

Summer can go a couple different ways, and it may affect the overall timeline of my thesis year. I’m preparing to once again apply to the Imagineering Show Programming internship in California. However, this year the internship runs from June to December, which means I would need to miss Fall 2023. If I am selected for the internship I will definitely accept that position and work with my advisors to devise a new thesis year schedule. If I do not get the Imagineering position, I will be looking for remote internships or positions here in Columbus. My thesis year would then continue on the normal schedule by starting in Fall 2023.

Emotional/Mental Health Check-In

Overall I’m feeling excited and prepared for the semester. I’ve arrived at a place where I’m comfortable with my thesis selection, and I’m tying up the loose ends in my knowledge to help me enter thesis year with a wellspring of knowledge and practice. My 2 major goals for this year are to create something with the microcontroller system, and get a simple test done in the Oculus pro headset. These are big goals but I believe I can definitely accomplish them with the help of my committee and my peers. I’m also very happy with the state of my research writing because I have a lot of material that will be ready for integration into my final paper. I am constantly referencing my annotated bibliography so I’m very happy I’ve created and continue to create entries for that document.

Here’s to 2023!

Hi there!

Thank you so much for visiting my graduate research space! This website includes a current MFA research blog, project documentation, research explorations, writing, and more! You can find all blog posts here on the home page by scrolling down, searching for a post or term, or browsing posts by month.

Introduction

The semester is finally coming to a close! I’m currently wrapping up a couple of projects and research explorations so let’s talk a bit about what has changed recently.

1. I’m finishing up my Puzzlr project and have been making great progress on the circuit system and design elements. I’m currently in the refining and digital component stage. I hope to nail down and finalize the puzzle board this week, and then work on the simple interface that will accompany the puzzle.
2. My haptic glove exploration has revealed some very important things to me and I’ve made some key decisions for my future thesis project.
3. I’ve begun experimenting with the Arduino Bluefruit system and am getting started with some tutorials and circuit python projects!
4. I’ve secured my first thesis committee advisor, Dr. Matt Lewis, and will be asking 2 more professors this week to join my committee! I ‘ve also begun writing my thesis preface and have made some awesome progress in my overarching literature review.

Haptic Glove Update

So after wrestling and fighting with the Prime X haptic glove system, I’ve made the difficult decision to move away from this system. The gloves work really well for certain things, but there is a lack of documentation and consistent support to implement them into game engine systems. I conducted several tests evaluating the interactions between the Manus system and the HTC Vive tracking system and found that the relationship between the two causes issues with the tracking data. In addition to this, I also had the chance of testing out the Oculus Quest Pro, which was an amazing experience and surpassed my expectations. The reason I had started with the gloves was because the hand tracking on the Oculus Quest 2 was not the best, but on the new Quest Pro the hand tracking is amazing. It is fast, accurate, and does not require the HTC tracking system. Because of this, I’ve decided to proceed with the Quest Pro and I’ll be creating a small exploratory project next semester to refamiliarize myself with Oculus development within Unity. This means the haptic element of my thesis will have to live on the within the microcontroller interaction systems that I create. As of now, I’m thinking on 3-4 simple interactions: a large button, a large switch, a haptic surface that would turn into a screen, and a dial. These interaction methods would provide different and unique interactions to stimulate the guest and enrich their surroundings. I’m super excited to start developing for the Quest Pro, and I look forward to making some exciting projects.

Arduino Bluefruit: Embarking on a New Journey

After experimenting with the Makey Makey and seeing how far I could take it, I’ve finalized my decision to use an Arduino system for my future thesis. The reason behind this decision is that the Makey Makey is far too unreliable to be used in such a complex system. The board itself is great, but lends itself to connection and implementation issues. There would be a lot of cut-and-paste wiring, and the primary method of interaction would require me to create my own circuits from foil and glue. So in comes the Arduino Bluefruit. After discussing with Dr. Lewis, I have begun experimenting and learning about the Arduino system and am currently exploring circuit python tutorials and projects to see how I can best use this system for my thesis. The use case for my thesis will be simple: create some buttons and switches that correspond to input within the experience. This would likely be routed into Unity as keyboard inputs or through a script that allows me to plug the board outputs into the game engine. From now until the end of next semester I will be exploring the Bluefruit and learning more about Python and Circuit Python through small projects and a potential collaboration with Thomas Ellsworth and Mila Gajic.

Thesis Updates!

I’m happy to share I’ve secured my first thesis advisor Dr. Matthew Lewis. Dr. Lewis will be mentoring me and guiding me on the overall completion of my thesis. I asked Dr. Lewis because of his extensive computer science skills, as well as his passion for research and innovation. I plan on continuing to form my committee with Shadrick Addy, and Alex Oliszewski; they will be providing guidance on virtual reality experiences and themed entertainment respectively.

I have also begun writing my preface, and am well into the process of writing out other elements of my thesis. I completed a 25-page committee proposal that included a concise thesis question and research topic, a definition of my 4 key elements, an extensive literature review, and a list of questions I’ve been asking and hope to answer through my research. I also see this committee proposal as an excellent way to get some thesis writing done ahead of time!

Where to Next?

So there will most likely not be a December week 1 update because I will be leaving for Winter break and will be focusing on family and rest as much as I can. However, I will be visiting several theme parks during the break so I also plan on taking some notes and observations about certain accessibility elements and inclusive design techniques at those parks. The next update will come in early January when I embark on my new projects and research endeavors for the coming semester.

So what’s the plan!?

A Microcontroller Project

A Small Oculus Pro Experience to Test Hand Tracking

An Exploration of Python and Circuit Python

Thesis Production Plan

See you next year!

Thesis Explorations

I realized I was getting slightly disorganized with my posts so I wanted to clarify some stuff. The thesis exploration projects I’m doing this semester are Puzzlr and Glove Test 1. Puzzlr is the project I’m working on in my devising experiential media course, and Glove Test 1 is the project I’m working on in my Design Studio course. Both of these projects are contributing to my final technological explorations and will eventually be combined next semester. You may also find Cycle documentations for Puzzlr on this blog and they are specific posts dedicated to the progress made on Puzzlr. 

So about Glove Test 1, 

In glove test one, my main goal is to get the Manus Prime X Gloves to work in Unreal engine with a simple physical surface-based interaction. This will be done by combining the Manus system with the HTC Vive pro trackers. So far, I’ve gotten the gloves to work in Unreal engine, and my next step is to get them working in VR. When I launch the VR preview, the fingers move but the wrists stay put. Originally, I had been planning to use the Manus Dashboard system to combine the HTC Vive tracker data and assign it to the wrists, but that has proven more difficult than I expected. The entire calibration process within Manus Dashboard is tricky, and doesn’t work very well. Because of this, I’ve decided to instead overlap the data within Unreal engine itself, and assign the tracker data being collected in unreal to the wrist position of the player’s hands.

So on Wednesday the 9th, I combined the glove data from the Manus dashboard and then assigned the 2 Vive trackers to the right and left hands. Once I started up the test it ran well and the hands in unreal were moving and bending realistically! Now that another obstacle has been overcome, I will now work on tying the hands to a VR pawn so they orient correctly around the headset when someone plays the experience.

Puzzlr Update

After a week of manufacturing, sanding, an tweaking, I’ve successfully refined the final white acrylic prototype of the Puzzlr board. I’ve now moved onto creating and editing the wire setup to get everything connected and ready for a playtest!

Overview

This week I’ve made some great progress on a variety of different elements regarding thesis exploration and class projects. I’ve broken down this check-in into a couple sections that detail my progress in specific areas.

Thesis Explorations

So, this week I’ve been watching and exploring some tutorials and materials to help my transfer my knowledge of the Manus Haptic Glove system into Unreal engine. I am familiar with game development in Unity but this will be my first time developing inside of Unreal Engine. The system is fairly similar, but there are some key differences in the way Unreal engine works that are slightly challenging to learn. Nonetheless, I’ve set out to start learning some of the basic components of the software to have a working simple haptic gloves experience by the end of this semester. The Manus Gloves system works by bringing in positional data from the Manus Dashboard program into the game engine of choice. Then, you take the positional data and retarget it to the mesh/skeleton of your choice. In my case, I’m targeting the data onto a simple set of hands that will be present in my experience. Once I get this working, this will have solved a huge issue I’ve been having in my development process.

In addition to progress on my exploratory progress, I’ve also made great progress on my committee proposal. I’m about 15 pages in and plan to deliver my formal committee requests by this weekend!

Class Projects

This week I also made some excellent physical progress in my Makey-Makey puzzle system. I’ve already laser-cut a new prototype, and will soon begin integrating the circuit into the new puzzle board. The new board is also printed in opaque acrylic, so the pieces feel sturdier and more grounded than the previous thin wooden pieces.

Project Brief

For the final project in my Devising Experiential Media Course, I’ve decided to create a fun and challenging puzzle that will stimulate the player in a variety of different ways through 3 distinct play modes. The puzzle consists of six pieces arranged in a hexagonal shape. The goal of each level will be to complete the puzzle each round while navigating the changes in the sensory elements of the puzzle. My research thread focuses on microcontroller inputs and the conceptualization of touch, so I see this project as a way to explore creating micro-interaction systems through physical objects.

The first round of the puzzle will be basic; the player just has to complete the puzzle by getting all the pieces onto the board in the correct configuration. The contacts on the bottom of the pieces and the board itself will give a slight hint as to how to approach the puzzle (more on that later). There will be both a visual and auditory feedback component informing the player about their progress.

The second round will involve covering the players hands so that they can only use the visual/auditory feedback from the system rather than looking at their hands. The pieces will also include unique textures that will then be mirrored on the screen to guide the player on what textures go where on the board.

The third and final round will involve blindfolding the player so they will have to rely on the textural elements of the pieces coupled with the auditory feedback. In this section, the auditory feedback for each piece will be unique so players can keep track of what pieces go where. This final level is intended to be very difficult and will be used mostly as an experiment.

Production & First Prototype

For the first phase of iteration I created a schematic and then used it to laser cut a preliminary model of the puzzle and circuit housing. After laser cutting and setting everything up I realized some parts of the first prototype need to change:

1. The engravings on the bottom plate are too narrow and can’t house the wires and cables necessary for the puzzle piece contacts.
2. The engravings on the bottom of the puzzle piece are on the wrong side of the pieces because I forgot to flip them before printing
3. The circuit layout is confusing, and requires more thought into how the ground/input wires are going to be set up in the final board.

After thinking about those changes I modified the schematic and created a new one featuring a new build layout, a new circuit component, and a new circuit layout. I decided I would create two levels to the puzzle, one for the circuits and one to put the puzzle pieces on. This allows more room for cables and components under the puzzles, and ensures a clean look on the top. The new circuit component is a little adapter (in the middle of the schematic) that will serve as a bridge between the circuit housing and the top of the puzzle. The component will take the ground and input cables and attach them to two pieces of foil that are sticking out at the top, the puzzle piece will then have a piece of foil underneath that, when placed on the contact, will complete the circuit.

Next Steps & Connection to Thesis Research

The next step in the project is to make the second prototype and get the circuit laid out and working for the next cycle! This connects to my research thread by allowing me to explore and get comfortable with microcontroller systems. I’m experimenting with fabricating physical interaction methods that will translate into digital inputs and connections for my future thesis experience. I plan on combining what I learn in this project with what I learn in my explorations of haptic gloves into a final experiment next semester before beginning thesis production.

After several hours spent digging through documentation, experimenting with the plugins, and installing the right content, I’ve finally gotten to a place where I feel like I’m making some real progress. I’m currently in the process of setting up the Metahuman and individually reassigning every bone in the arms and hands to get the correct movements and translations from Unreal. It’s been a tough journey so far but I’m getting closer to having some real content to show!

Enjoy this picture of this painstaking and annoying process!