I would first draw the distinction between a private company and a public company.  If you start your own company tomorrow, say taking some money you have saved, and opening a store on the high street, or maybe selling a new piece of software on the internet, you have started a business, which you own yourself--privately.  You have to make some choices as to how the government and the rest of the world treats you from a legal and a tax standpoint.  You could just run the business as yourself, InWales, and report the money you receive from your customers as income, and deduct the money you spend as expenses to attain that income, and report it on your personal income tax form.  And legally you and your business could be one in the same--ie., if you do something someone views as wrong, they could sue you, and not just your business--ie. if you lose the lawsuit they could lay claim to your personal assets, such as your home (depending on various laws in the country you live in).  But clearly, you own your business yourself.

You could also take a step, which is different country to country, and set up your new business as a corporation.  This would separate yourself from your business to some extent.  It may mean that from a tax standpoint, your business reports sales and its profits, and the business is taxed by the government under different rules and taxation for corporations.  And then you personally would have to also pay taxes as an individual, just like you already do.  So your business would pay you a salary for example, and then you as an individual would pay taxes on the salary.  The corporation also distinguishes the business from you in a legal sense.  So you may be able to separate your personal belongings (example home) from your business, and if someone sues the business, the business is liable. If someone sues you personally you are liable.  But it's possible that the business could lose the suit, and you might be protected from losing your home.  Very sticky point, because this varies so much from country to country.  But you still own the business, so even though you now call it a corporation, it is a business owned by you as a private individual.

Maybe you bring in a partner, who contributes money and works for the corporation.  You and he/she decide he owns part of the company.  So you share ownership, often called equity, of the company.  Maybe you bring your family on board later, sons and daughters, and you allow them to also become owners, by contributing their labor,,,,or perhaps by buying into the company, say if your brother earned money somewhere else, and wants to join you.  You agree to give him 10% ownership, equity, for his money.  But the company is still private, owned by private individuals.

Say you are wildly successful, and you opened a store on the high street, and you see that you can expand nationally, opening stores on lots of high streets.  but you need cash to buy land, build stores, etc.  you could borrow money from a bank, and open one other store for example (and many small private companies do exactly that).  But, you think your idea is so great that you want to open 100 stores (and I also want to get to public equity), and the bank rolls its eyes and tells you no way they are loaning you enough money for 100 stores.  But you are excited and really want to do this, and get the money and do it fast.  And to get that money, you are willing to trade some ownership in your company to other people in return for their cash.  You can get lots of cash this way, if new potential owners agree that you have a great idea that will earn money,,,but these new owners will own the company with you,,so you'll begin to lose complete control of your company.

One place to get this money reasonably quickly, is from a venture capitalist.  This person exists to work with people like you,,,,providing money in return for ownership,,,,and hoping to grow his money.  He normally has put together his own "private company" (usually called a partnership), and he has people who have given him their money, to invest on their behalf.  They have likely seen him do investments like yours in the past, and they've been successful--so he has a track record of success,,,,taken $1 million from several people, and five years later given them $2.5 million back.  But he and his "partnership", are organized privately--they sign contracts with each other as to how much money the "venture capitalist" will earn,,,,and how much he will make if he invests their money wisely and gets a great return for the other partners and himself.  (over simplifying here, but conceptually on course).  Historically the venture capitalist works with small start up businesses, thus the name venture, and provides money, sometimes called capital, to entrepreneurs like you,,,in the hopes of both of you creating something new out of this partnership.  (Basically Google started very much like this.)  

So your company started out being private and owned by you.  Grew to include friends and family,,,still private and owned by all of you.  You all owned all of it,,,you owned 100% of the equity.  then you needed big money to grow your business fast,,,from people willing to take a risk,,,so you found these private partners, called venture capitalists (VC's).  Now all you private people own all the company together--you own with the VC's, 100% of the equity of the company.  So you are still in the world of private ownership, or private equity,,,,but you have morphed into a larger group of owners.

Venture Capitalists are buying equity in early stage companies with the financial motivation to earn a profit.  They are making investments that are generally risky, because they are so early stage, so they expect a number of their investments to fail, and in these failures they would lose all of their investment.  However, they raise money from their partners in "funds', collecting commitments these days for $100 million to as much as $1 billion or more.  They would hope to invest in 10 to 20 companies, hoping that their winners will more than offset their losers, and thus the overall fund, or monies raised, will see a good return on investment.  The startups that are winners will need to have what is called an "exit event", which means the startup company will be sold to a larger company, so the VC's can return monies and hopefully a positive return to the investors.  The other option for an exit event is for the company to "go public".  This means that the company offers shares, equity ownership, to the public on the public stock markets, such as in the US the NASDAQ or New York Stock Exchange.  (The original founders of the startup, such as you in our example, know this of course, and you may stay with the company and be the CEO through all of this, or you may decide running a big company like this is not what you really want to do.  But all of that presumes success of course.  As you might imagine there are many failures and many great success stories--many who end up loving the whole venture capital process, many who end up hating it.  This often correlates with the success of the venture--I would imagine the Google founders are pretty happy with the process, though they are not the CEO today,,,,but certainly are incredibly wealthy.)

Often the "going public" route is a good one not only from the standpoint of giving the investors a chance to cash in on their investment, but also because the startup company may need more money to fund their business.  As an example, this is often the case in a bio-tech company, which is developing a new drug.  It takes years and $100's of millions to develop a new drug, conduct the clinical trials, build manufacturing, hire a sales force, etc., and of course many of these companies fail.  Just as an example, there is a company called La Jolla Pharmaceutical Company which is today in the hopefull final phase, Phase III of a new drug for a disease called Lupus.  

La Jolla Pharmaceutical Company is dedicated to improving and preserving human life by developing innovative pharmaceutical products. The Company's leading product in development is Riquent®, which is designed to treat lupus renal disease by preventing or delaying renal flares. Lupus renal disease is a leading cause of sickness and death in patients with lupus. The Company has also developed small molecules to treat various other autoimmune and inflammatory conditions.

A little more about Lupus from a recent 10K filing by the company:

Lupus is a life-threatening, antibody-mediated disease in which disease-causing antibodies damage various tissues. According to recent
statistics compiled by the Lupus Foundation of America, epidemiological studies and other sources, the number of lupus patients in the United
States is estimated to be between 500,000 and 1,000,000, and approximately 16,000 new cases are diagnosed each year. Approximately nine
out of 10 lupus patients are women, who usually develop the disease during their childbearing years. Lupus is characterized by a multitude of
symptoms that can include chronic kidney inflammation, which can lead to kidney failure, serious episodes of cardiac and central-nervoussystem
inflammation, as well as extreme fatigue, arthritis and rashes. Approximately 80% of all lupus patients progress to serious symptoms.
Approximately 50% of lupus patients will develop kidney disease which is a leading cause of death in lupus.

The company was founded in 1989.  It has spent more than $250 million to date, and still does not have a drug approval, therefore, no sales and no profits as of this time--obviously not a Google.  It is very difficult to raise that much money from private investors, for a risky startup company.  So often a bio-tech company will raise early stage money from VC investors, hoping to discover molecules, develop science that shows the drug may have some applicability to solving certain diseases, and make some initial progress on the long science and clinical trial program needed to get an approval of the drug.  At some point they may develop the company to a point that they can present the company's status on their development, identify the opportunity if successful, and then attempt an "Initial Public Offering" to raise funds to complete the trial.  The nature of the risk of the investment will be clearly laid out in the offering documents, but so will the size of the potential markets if the drug is approved and successful.  So some public equity market investors will want to include a high risk/high reward stock such as this in their portfolio.  La Jolla to date has not been successful in gaining approval, as their first attempt at a phase III approval failed, though there was significant positives out of the trial to encourage the company to raise more money, and investors to put more money in, in hopes that this current Phase III with an improved, strengthened product and a better designed clinical trial will be successful.

But a company like La Jolla was far to risky of a proposition for the public markets in its early days.  They could not have raised money until they had shown significant reason to believe they might have a success.

Once you are in the public market, your company is heavily regulated in terms of financial reporting requirements on a public basis.  You are owned by public investors, and there are reams of documents and legal requirements for reporting.  The requirements are far, far, far less stringent for a company that just has private investors.  Not that fiasos don't happen, because there are crooks everywhere in life, and Enron and Tyco show that is true in the public equity markets.  But investors are far more protected in these public markets than they are in private markets.