Middle School Project

Welcome to the “Future Innovators: Cool STEM Projects to Change the World” series! In this project, we’ll create a fun and interactive Augmented Reality (AR) Environment Explorer using Scratch. You’ll learn how to build a virtual experience where players use their webcam to interact with real-world objects. In your game, digital objects like animals or plants will appear on the screen when certain markers or actions are detected. It’s a cool way to blend the real world with the digital one—without needing complicated tools or advanced coding!

By the end of this tutorial, you’ll have built a virtual environment explorer that uses basic AR principles to identify objects and display digital images, all using Scratch’s camera features and event-based programming.

What You Will Learn

  • How Augmented Reality (AR) can combine the real world with digital objects.
  • How to use Scratch’s camera features to detect objects and interact with them.
  • How to build fun AR-style games using event-based programming and simple movement blocks.

STEM Learning Process

1. Science: Understand how cameras capture real-world images and how computers use them to display virtual objects.
2. Technology: Learn how to use Scratch’s built-in tools to create interactive and augmented experiences.
3. Engineering: Build a game that combines real-world interaction with virtual graphics.
4. Math: Use simple coordinate systems to position objects in your AR world.

What You Need

  • A computer with internet access and a webcam.
  • A Scratch account (free): Sign up at scratch.mit.edu.
  • Printable markers (like QR codes or simple shapes) to act as AR triggers (optional but fun for added realism).

What is Augmented Reality (AR)?

Augmented Reality (AR) allows us to blend the real world with digital elements. Think about Pokémon Go, where creatures appear on your phone screen while you’re walking around your neighborhood. AR uses the camera to show the real world but overlays digital objects on top of it. In this project, we’ll create a simple AR environment explorer where digital objects appear based on what the camera sees or what actions the player takes.

Step-by-Step Guide to Building the AR Explorer in Scratch

Step 1: Sign Up for Scratch

If you don’t have a Scratch account yet, visit scratch.mit.edu and sign up for a free account. This will allow you to save and share your projects.

Step 2: Create a New Project

Once logged in, click Create in the top-left corner to start a new project.

Step 3: Choose Your Explorer Sprite

  1. In Scratch, sprites are the characters and objects in your game. For your explorer sprite:
  • Go to the Sprite library and choose a sprite (you could use a person, a robot, or even an animal to act as the explorer).
  • Rename your sprite “Explorer” to make it easier to organize.

Step 4: Setting Up the Camera Interaction

Let’s set up the camera so that the player can see themselves in the background as they interact with the game.

  1. Go to the Video Sensing section in the Scratch blocks.
  2. Drag the “Turn video [on]” block onto the workspace.
  • This will turn on the webcam and display the live video feed in the background of the game.
  1. Set the transparency of the video feed using the “Set video transparency to [50]” block so that the virtual objects (like animals or plants) will be visible on top of the camera feed.
  2. Use the “When [green flag] clicked” block (from the Events section) to trigger the camera setup when the game starts.

Your code should look like this:

Plaintext
When [green flag] clicked
    turn video [on]
    set video transparency to [50]

This makes sure that the video feed is on when the game starts, and the virtual objects are overlaid on the camera feed.

Step 5: Adding Virtual Objects

Now we’ll add the virtual objects that the player can “discover” while exploring the environment. Let’s start with adding a virtual tree.

  1. Choose a sprite from the Sprites Library (or draw your own tree, animal, or object).
  • For example, choose a Tree sprite or draw one in the Scratch editor.
  • Resize the sprite to make it smaller or bigger, depending on the game’s environment.
  1. Place the tree in the game world by dragging it into position on the screen. We’ll later make it appear based on certain actions, but for now, just place it where you want.

Step 6: Making the Tree Appear When Triggered

We want the virtual tree to appear when the player moves into a certain position or when a marker is detected. Scratch can detect when a part of the screen changes (e.g., when the player waves their hand over a certain area).

  1. Use the “Video Sensing” blocks to detect motion.
  • Drag the “Video motion on [this sprite]” block into your workspace.
  • Set the motion amount to trigger the tree’s appearance when the player moves in front of it. For example:
Plaintext
When [green flag] clicked
    hide
    forever
        if  on [this sprite]
            show

This will hide the tree at the start, but when the player waves their hand or moves in front of it, the tree will appear on the screen.

Step 7: Adding More Interactive Objects

Let’s add more virtual objects to create a rich AR environment. You can add:

  • Animals: Choose animal sprites (e.g., a bird or a fox) that will appear when the player interacts with certain areas.
  • Buildings: Add buildings or other structures that pop up when the player moves into a specific part of the screen.

Making Objects Move

To make the game even more interactive, you can add movement to the virtual objects when they appear.

  1. Select an animal or object sprite.
  2. Use the “When [flag] clicked” block to start the game, and then:
  • Use a “forever” block to make the animal move continuously.
  • For example, make a bird fly back and forth across the screen:
Plaintext
When [green flag] clicked
    forever
        glide 2 secs to x: [200] y: [50]
        glide 2 secs to x: [-200] y: [50]

Step 8: Adding Fun Facts About the Environment

Let’s add fun facts that appear when the player “discovers” a new object (e.g., a tree or animal).

  1. Create a new message block to trigger when an object appears.
  • For example, when the tree appears, use a “broadcast [message]” block to trigger the appearance of a fact about trees.
  1. Add a “say [ ] for 2 seconds” block to make the explorer or another sprite display a fun fact when the object is discovered:
Plaintext
When I receive [tree discovered]
    say "Trees help produce oxygen!"

You can do this for each object in your AR environment to make the game more educational and fun!

Step 9: Testing Your AR Explorer

Now it’s time to test your AR Environment Explorer!

  1. Click the Green Flag: This will start the camera and the game.
  2. Move around in front of the camera or wave your hand to see the virtual objects (like trees or animals) appear on the screen.
  3. Read the fun facts that pop up when objects are discovered.

Customizing Your AR Explorer

Here are some ways to make your AR Environment Explorer even more unique:

1. Add More Virtual Objects

  • Create an entire ecosystem by adding different types of plants, animals, and even weather effects (like rain or snow).

2. Create an Adventure Story

  • Turn your AR explorer into a story where the player moves through different environments and uncovers mysteries (e.g., lost treasure or hidden animals).

3. Add a Scoring System

  • Add points for each object the player discovers, and keep track of the score to make it a game.

What’s Next?

Congratulations! You’ve built an Augmented Reality Environment Explorer using Scratch. You can now explore a virtual world right on your screen, discovering objects, reading fun facts, and interacting with digital elements overlaid on the real world.

In our next project, we’ll build a 3D space exploration game where players fly through the universe and discover planets. Stay tuned for more cool and fun STEM projects released every week!

Resources and Additional Help

Categorized in:

STEM