Terrain Using SCMDraft2
Note: Since SCMDraft 2 is pretty much the de facto standard tool for advanced terraining (as of this writing), there's little sense generalizing this topic to cover other tools.
Isometric terrain: The standard option for terrain painting, and the one you'll (likely) be using the most. Little explanation needed here. You paint your terrain in pretty-looking and well-blended isometric shapes by selecting from the available terrain type list for your map's current tileset. But beyond that, you'll need more options. Each of a given map's tileset's Terrain types are constrained by which adjacent terrain types they're allowed to combine with. What this means is, for instance, the Badlands' water terrain type must always be next to dirt terrain. Trying to place, say, asphalt next to water (which itself requires adjacent dirt) will automatically create an intermediate layer of dirt before water can be allowed next to it.
Because of these constraints, some effects and functionality options can be limited. This, of course, is what advanced blending is for.
Rectangular terrain: The most basic (and crude) way to violate the standard constraints of Starcraft's isometric terrain styling is by directly painting rectangular tiles of the selected terrain type. You won't get any pretty isometric meshing here, but for strict functional use this option gets the job done. Little to talk about with this option, really, except to mention that rectangular tiles will NOT affect the 'flow' of placed isometric terrain, and will even be rudely overridden by the standard isometric blending. This is why it's a good idea to always paint your custom rectangular terrain after your final draft of your isometric layout.
Unless you plan to make your entire map with rectangular terrain, of course.
Subtile editing: This is a specialized editing mode for tweaking tiles of isometric blends. The usefulness of this mode comes in when you understand a bit about the nature of isometric tiles, which I'll explain later. In short, you'll use this option mainly only for isometric polishing and surface cleanup... if at all.
Tileset index: NOW we're cooking with fire! This will be one of your main weapons for just about all terrain blending you'd ever want to do. Unfortunately, there's no way around getting your hands dirty to master using this mode. Prepare to be sifting through mounds and mounds of oddball tiles individually looking for pieces that fit together. It's like a jigsaw puzzle, really.
Later, I'll gloss over some handy tricks for making your job easier with blending, though. And this isn't your only tool, either:
Brush palette: This tool allows the user to use a large "compilation" of blends put-together by an other terrainer. All you must do is select the blend you want and paste it where you want into your map. Moreover, you do not even need to open a new map, it's a sub-window opened into your map, just like when you edit triggers.
... The perfect tool for the lazy-boys of us
Copy/Paste: You just click and drag over any map tiles you want, press Ctrl C, and then Ctrl V, and click elsewhere to duplicate ad nauseum. Pretty basic. When applying blends on a large scale, you'll be using this a lot, so get the hang of it. Note that holding the Shift key and drawing multiple selection rectangles (even with overlap) can allow you to select non-rectangular portions of terrain for copying; invaluable for blending. There is also the option of pressing the Ctrl key to toggle free-draw selection of individual tiles, which may be useful once you get the hang of it, too.
Also, get in the habit of re-copying groups of multiple terrain pastings in order to paste them in large quantities. One-at-a-time pasting work can become a real time (and patience) sink, if you let yourself get into it.
Brush sizes: With each terrain mode (except Copy/Paste), you'll have the option of specifying the width and height of your paint brush size. This option should be pretty self explanatory. Make use of it whenever you can to cut down on as much tedium as you can when painting large areas.
Other tricks: Under the Doodads layer, you can select any doodads you've placed, right click on them, and select Convert to terrain to flatten them into their corresponding terrain tile and sprite components. Don't worry about what that means; we'll cover sprites later. The terrain tiles, however, can be then grabbed and used with Copy and Paste. This can remove a bit of the hassle with hunting down individual tiles for blending in the Tileset index dialog, if you're so inclined.
Additionally, Ctrl clicking individual terrain tiles under the Tileset index mode will seek out the selected tile within the tile list. This is a great, hands-on way to explore the list by using the map's own terrain tiles as a reference for understanding the confusing tileset list. You'll be using this a lot when toying around with or inventing new blends.
Finally, to aid in tile placement and visualizing, you can toggle default map grid by pressing Alt G in any mode. Mainly you'll use grids (combined with the 'Snap to' options) for aligning any units and sprites you'll need, but having guide lines drawn for tile placement is also handy for tricky or large-scale blends. Alternate grid sizes can be invoked by pressing Alt U (ultra fine; 1/8 of a terrain tile), Alt F (fine; 1/4 of a terrain tile), Alt L (large; 2x), Alt E (extra large; 4x). Custom grid sizes is also an option under the Options -> Grid menu.
Credits
Thanks to Tuxedo-Templar for writing this in November 2009.