For the animation Seeing Red I have been developing a "crosshatch" shader for Mental Ray. I know it is bad to refer to something by what it is not, but I haven't seen a better term for "Nonphotorealisting Rendering".
I was working in XSI 4.2, and wrote the shader and .spdl file to integrate it into the Render Tree. It is based on "Advanced Renderman"  chapter 16.3
Unfortunately polygonal the texture derivatives this technique is based on, are not defined in Mental Ray for polygonal objects . The objects in the image on the right are all NURBS surfaces. I tried pre-rendering the UV's and creating in texturemap (in camera space) with the derivatives in the color channels (storing the difference in UV values for a pixel as compared to its neighboring pixels) but got all kinds of weird artifacts.
Another measure for how big objects appear on screen in the distance to the camera (duh!). Using the inverse of that distance I got a working result . The orientation of a surface is also an important factor for the relative size of a texture on screen. The shader I wrote accepts incidence as an input to somewhat compensate for that, though incidence is unfortunately not related to the direction of U and V.
I rendered a test with a big camera move to see how the shader would behave. As in the image above, the outlines are created using the standard "Toon Shader" that ships with XSI.
Rendering with noise on the lines created jumps on the edges of the Render Tiles. Switching off Ray Tracing fixed that. To still get the shadows I pre-rendered the animation with simple lambert shaders on all objects. I did first try using shadowmaps but all the lights in the scene are directional and the shadowmap calculation really slowed down the rendering.
Quicktime movie (400kb)
Some of the weird lines, especially on the "trees", are the result of crappy UV mapping. For this shader to produce nice results the UV's should be evenly distributed across the surface.
Next test: Exaggerated Shading