I'll expand on the answer about coins/stocks/funds...
If you want to buy gold stocks or a gold "exchange traded fund" (ETF), then you'll have to open an account at a brokerage firm and deposit money before you can make the purchase. There are several national reputable online firms you can use, such as TD Ameritade, E-Trade, Schwab, Fidelity, Scottrade and Tradeking. To open the account you'll need to complete an application and provide address, social security number, bank information, financial information and prove your identity. To make a deposit you can send (or drop off) a personal check or have your bank "wire" the money. If you make the initial deposit with a personal check you probably won't be able to make the purchas until that check clears, usually several business days.
Using the gold ETF would be the most direct way to have a gold investment using a brokerage account. The price will move nearly perfectly with the actual price of gold. And while owning gold mining stocks can work quite well, there are so many other variables in the performance of a single mining stock that you need to do a lot more homework -- owning a gold mining stock isn't just owning gold, even though the share price will usually move with the price of the metal.
As for buying actual gold, your best vehicle is to buy coins -- the most price efficient are probably the various 1 ounce coins issued by many governments such as the US Eagle, Canadian Maple Leaf or South African Kruggerand. Each of these coins contain one ounce of gold and the price tracks very closely to the market price of gold, but you'll pay a premium above the gold price of about $100 at many dealers. (Gold today is at about $1295 per ounce, so one coin is likely to be almost $1400.) Many dealers will want you to pay cash and will barely ask for any information.
Finally, I urge you to think carefully about this because gold is at an all time high. Almost all investment advisers suggest that a portion of your investment money be in gold, but almost none would suggest that all your money be in gold. You should build a diversified porftolio. So if you start with gold, 1) be aware that it may very well decline from here, and 2) starting a plan to have some investments in stocks, both foreign and domestic, and some investment in more conservative "fixed income" securites (also called "bonds".)