PvE isn't generally what's advertised or talked about but it is there. In heaps. It's just that the PvE isn't really unique in the same sense that the PvP is.

General PvE consists largely of:

1) Missioning. This is the most prevalent form of PvE, and is essentially randomly selected quests. They come in different forms, such as trade missions, courier missions, combat missions, mining missions, and storyline missions. Of these combat missions are the most rewarding, most popular, and generally the most fun. They are also, barring other players, the only ones that involve any danger.

Storyline missions come in the form of the other 4 types, but are worth much more and have more profound impact on your faction standings.

Missions are given out by NPC corporation agents which are seeded throughout the galaxy. You essentially have to work your way up the corporate ladder by doing missions for lower level agents, then mid, then high, etc. There are a number of addition factors that influence the rewards, such as your corporate standing, individual agent standing, skills, and the security status of the system the mission takes place in. (Doing missions in low-security space pays more, but you will have to keep an eye out for other players who may wish to kill you.)

There are also Epic Story Arcs, which are fairly new to Eve. They are a chain of connected missions which are geared towards delivering a consistent narrative (Not just: Pirates have attacked transport X, go save it!) and generally end with a large pay-out at the completion.

Missions also supply you with Loyalty Points (LP) which can be redeemed at any station owned by the issuer's corporation for stronger ammunition, improved ship modules, blueprints, implants, faction ships (which are expensive as heck, but vastly superior) etc etc etc. Each corporation has it's own LP store, and offers different rewards.

As I said before, Missioning is # 1. It pays well, the danger is minimal, and the LP can be easily converted into valuable goods which can be resold.

2) Salvaging. Generally a mini-profession within missioning. With the right skills and equipment, you can salvage the wrecks of ships. While most salvage isn't worth much, you do accumulate a large amount of it, and the odd pieces are quite valuable. What's more, savage from more advanced player ships (tech 2 ships) is quite valuable. Salvage is used to create rigs, which are "permanent" modules that can be placed in a ship to boost it in one area at the expense of another (More armour, less speed, etc.)

3)Mining. The #2. It's not combat oriented, but it's a huge part of Eve, and with the right skills, ship, and mind-set (or a movie playing on a second monitor) it can pull in a hefty amount of ore. You can use this ore to build ships, modules, etc, or you can sell it on the market. While this is a huge aspect of Eve, it really not much to talk about.

4)Trading. Again non-combat, though some people do refer to it as "Market PvP." This is, for dedicated and patient players, an easy way to turn a very decent buck. There's a number of things that you can do... supplying needy markets, or snagging especially low-priced items and reselling them. Heck, I know a couple people who scan the contracts constantly, looking for items the seller forgot to add a couple 0's to, and then resell it for the proper price.

5)Exploration. This one, like missions, comes in a number of different varieties. Some exploration sites are pure combat, others have valuable asteroid fields/gas clouds to mine after you have cleared the pirates guarding them, others have derelict wrecks you have to hack (another mini-profession) to get at the goods inside them.

Exploration is more involved than mission running, and tends to be a lot more hit and miss. First off, you have to scan down the sites, which involves triangulating their position within a system using three or more probes. as you get pings, you move the probes around, decreasing their scanning radius and homing in on the signature. Sometimes you will come across a great multi-stage escalation with fantastic faction blue-prints at the end. Other times you will spend 40 minutes trying to pin down a single signature only to find out when you get there that another player is just clearing it out, or it's a gas cloud when you really wanted some combat.

Exploration also scales depending on the security status of the system the site is in. The more dangerous the system, the more dangerous (and much much more rewarding) the site generally is. One of the big pushes behind alliances controlling 0.0 systems is so they can monopolize access to particularly bountiful exploration sites.

6)Wormholes. These are another newer addition. Wormholes can be found through exploration, and lead to "unknown space," which serve as isolated systems with limited and constantly fluctuating access points. Wormholes have a mass and time limit before they collapse, which generally regulates the size of ships (and the number of) which may pass through them, with lower-level wormholes not allowing much through, while larger ones can support battleship fleets.

Wormholes are quite profitable if you do it right. The NPCs you face (called Sleepers) have enhanced AI and other tricks which make them more dangerous, but the salvage and items they might drop cannot be aquired anywhere else and are worth a considerable sum. Additionally, wormholes have unique gasses to be harvested, and large amounts of valuable asteroids.

A lot of corporations like to set up shop in wormholes because of how isolated they are, but they come with inherent risks. If you're there with some buddies and the only guy packing a scanner/bookmarked the exit gets taken out, the rest of you are stuck until a) some stranger with a scanner happens by and takes you to the exit (extremely unlikely) or you all self-destruct. So always remember to bookmark the exit!

Additionally, unlike the rest of space, "unknown space" doesn't give you a list of the players currently in system with you. You might not realize there are others about until they have you locked and jammed.

7)Industry/Research. I won't go heavily into this as it is what you would expect. You can build anything if you have the right minerals and a blueprint. Researching involves upgrading blueprints to speed production, or reduce the needed minerals.

And, quickly:

8)Ratting. Another combat oriented form of PvE. Asteroid belts are often guarded by a small number of NPC pirates (or "rats," hence "ratting"). Ratting entails looping from belt to belt and destroying any pirates you find for their bounties. This is only really worth it in 0.0 space, where you face pirate battleships. In high security systems you can make more money doing almost anything else.

There's more than that, actually, but I've gone on long enough.