Making noise doesn't have to be hard or expensive.
Chances are, you have a whole ensemble of noise instruments hidden in your home.
Before you go broke at the music store, try out these simple tricks to make noise with junk:

• CD Aleatory Remix: With a sharpie pen, Draw lines on the back of a CD you don't want anymore.
  The CD will skip and reshuffle the music in novel ways.
  
• Radio Tuner: When you fine tune an AM radio, you'll find drones and pulses that feel at home in any industrial soundscape.
  
• High Pitched Terror: Rub styrofoam on a wet glass plate.
  The faster you rub, the higher the pitch.
  Works well on car windows.
  
• Tape Loop: If you've got an old cassette player, you can make your own loop center.
  Just open the case of a cassette, and remove the tape.
  Then cut just enough tape off to wind around the spindles and tape the ends together.
  Search "diy cassette loop" for diagrams.
  
• Walkie-Talkie Theremin: Tune two walkie-talkies to the same frequency.
  Then hold them a few inches apart and press one of their talk buttons.
  The resulting feedback can be played like a theremin.
  Move the handsets farther apart to lower the pitch.
  
• Cheap Maracas: Fill a pill bottle with fine gravel.
  Screws work just as well.
  
• Noise Toys: Electronic toys make some of the strangest sounds on their own.
  But the best noise comes from "circuit bending" the toy.
  Anyone with a soldering iron can learn how to circuit bend battery-powered toys into rugged noise instruments.
  Search "circuit bending" for detailed instructions.
  
• Candy Wrapper Whistle: Any candy wrapper or thin blade of grass can produce ear-splitting whistles.
  Hold the wrapper horizontally, and pull the edges taught.
  Press the corners of your lips together and leave a small opening.
  Now hold the outstretched candy wrapper in front of your mouth and blow hard.
  
• Cheap Microphone: Computer speakers and headphones become microphones when plugged into an amplifier's input.
  In most cases, the headphone's jack won't fit in the amplifier's input, so keep a 1/8th inch to 1/4th inch headphone adapter on hand.
  

If you find your closet barren of these household items, a trip to the thrift store will supply you with all the raw materials you need.
Finally, don't ignore free, open source audio software as a source of noise.
Audacity is a must for databending and everyday audio recording, and you can download it for free.