Diagram of the day #8

Is it me or are those sperm?

( from http://www.scratchapixel.com/lessons/3d-advanced-lessons/volume-rendering/volume-rendering-for-artists/ )

Raspberry Pi

Though i'd jot down some notes about setting up our kid's Raspberry Pi. Its been a bit of a slog due to waiting for cables, adapters, and whatnot to arrive. Still need to get a usb hub to make it able to connect to wifi

Here's the HDMI to VGA converter I needed to purchase to connect it to our old-ish DELL flat monitor that lay discarded for years in a corner and was very nearly given away due to under-use.

http://www.amazon.co.uk/gp/product/B0088K7QUQ/ref=oh_details_o06_s00_i00

/boot/config.txt had to have one of the following (can't remember which now)

disable_overscan=0
hdmi_drive=2
config_hdmi_boost=4

or

# Settings for Flatron VGA monitor
# 1024 * 768
# Force HDMI signal even if monitor not detected
hdmi_drive=2
hdmi_group=2
hdmi_mode=16
hdmi_force_hotplug=1
disable_overscan=0

which took several turns of switching between our HDMI tv and the monitor to get it working !!

Diagram of the Day #7

Q: How much T does it take to fuck with your ability to draw a triangle ?

A: 100 cups

( p.20 of http://kmh-lanl.hansonhub.com/talks/maxent00b.pdf )

How annoying is grep ?

So in a directory with only subdirectories, I was doing

grep -r 'Distribution2D' *.cpp

but it didn't work. "Bloody recursive flag is a load of bollocks" I thought to myself. So it turns out it does work but you have to be a bit careful with the file wild cards to make sure it finds everything into which it can recurse.
I.e. this worked fine.

grep -r 'Distribution2D' */*.cpp

Well, f**k me!

Diagram of the Day #6

You've heard of a bell curve. Well this is a bellend curve.



Also from the same paper, bottom right is an interesting shape... where have I seen that shape before ??




http://www.cv.ci.ritsumei.ac.jp/~takeday/research/paper/egmm04.pdf

Funniest website of the week/month/year?

http://damnyouautocorrect.com/category/best-of-dyac

Diagram of the Day #5

Its wang time again. This wang is importance sampled!
Woooo. No wonder that eye is looking at it.
Bucking the page 3 trend, this comes from p.13 of the siggraph 2010 "IS in production rendering course notes". Good work!

Diagram of the Day #4

This one comes from p.3 of :

http://langley.atmos.colostate.edu/courses/at622/lectures/supp_lecture_4A.pdf

iomanip

C++ std iomanip is super handy!
e.g.

"setw(4) << setfill('0')"

http://www.cplusplus.com/reference/iostream/manipulators/

Diagram of the Day 3

thanks to @fxmonkeydan for this little beauty :

these just make me laugh...

 


this week on Jamie Oliver inside a badgers arse :
"Inside a Badgers Arse Risotto".

RSL shadeop plugin memory

So there are 5 ways to manage memory allocation in an RSL shadeop.

1. per plugin init - lifetime from first call to end of frame, not threadsafe, no access to args

RSLEXPORT RslFunctionTable RslPublicFunctions(myFunctions,
                                             initStringBuffer,
                                             deleteStringBuffer);

2. per function init - lifetime from first call to end of frame, not threadsafe, access to args

static RslFunction myFunctions[] = {
   { "string appendTx(string)", appendTx, initStringBuffer, deleteStringBuffer },
   { NULL }

3. Set / GetGlobalStorate - lifetime from first call to end of frame, not threadsafe

4. Set / GetThreadData - lifetime of thread, threadsafe

5. Set / GetLocalData - lifetime of current shaded grid, threadsafe

Not bloody makefiles again ?

Replacing symlinks at compile time, so that republished shadeop DSOs don't fuck with existing shaders:

Makefile :
# here's the symlink path
MY_SO = $(PRMAN_PLUGDIR)MY.so

# resolve that to what the symlink points at
_MY_SOA := $(shell readlink $(MY_SO))

# double slashes in the path will mess up my shader macro's, so remove them
_MY_SO := $(subst //,/,$(_MY_SOA))

#now we can pass that into SL file scope using the pre-processor define flag, -D
shader .... -D_MYSO=$(_MY_SO)

Eat it, Makey - in your face, ha!

SIGGRAPH 2010

I am flying off to LA to attend my first ever SIGGRAPH conference tomorrow. I couldn't be more excited - its something i've always wanted to do since discovering SIGGRAPH papers existed when I was studying computer graphics at UMIST. ( 15+ yrs ago ! ).
I'm not sure why exactly but i've always been a fan of Jim Blinn's articles and work, and always thought it would be quite cool to attend a talk of his or meet him at Siggraph perhaps.
Any road up I am also due to give a short presentation at Pixar's exhibition booth about rendering the Olympus Map room clouds for Clash of the Titans. I am experiencing terror and excitement in equal measure. This is a new experience, usually its just the terror on its own. Fingers crossed it all goes to plan.

Diagram of the Day 2

Todays winner comes from the cutting edge of towel research.

( again, page 3 seems a popular choice ) :

http://reference.kfupm.edu.sa/content/a/b/abstract_a_layered_method_for_realistic__76571.pdf

Diagram of the Day 1

How they ever managed to sneak a picture of a cock and pair of balls past the moderator, i'll never know...

http://www.cemyuksel.com/research/dualscattering/dualscattering.pdf

( from page 3 ):

Makefile and GLSL

So it turns out, as cool as GLSL is, there's no support for #include directives.
Arse.
Includes are quite handy so they are.

Right, so what we need now is some existing pre-processor to expand the includes in-place.
What about gcc -E ( or g++ -E ) ? Weeelll, they are ok, but they leave place markers everywhere, it doesn't read very nicely. Also you'd have to name all your glsl shader files .c or .cpp or something icky like that.

Turns out you can run "cpp -P" and that will pre-process the includes nicely. It works on any old file name too - so no problem calling them .glsl
How do I automate it though ? A script ? A makefile ?
Well, since you're preprocessing code, its near enough to a compile to use a makefile.
This is where the (un)fun begins... actually lets cut a long story short, i'm already sick of makefiles without recounting blow by blow details.

basically you'd do this:

GLSL_SHADERS := $(wildcard hwshaders/*.glsl)
GLSL_SHADERSOUT:= $(GLSL_SHADERS:.glsl=.glslo)

%.glslo : %.glsl
  @echo $< $@
  @rm -rf $@ ; cpp -P $< $@

glsl : $(GLSL_SHADERSOUT) 

and then run

make glsl

Which would expect the glsl files to be one directory below the makefile in a dir called hwshaders.
Also I found out that if you are using a makefile conditional if, then the @ thingy to silence command echo doesn't really work. What did work for me was using the .SILENT rule.

.SILENT: safepub

safepub:
make clean
make
echo
if test -e /some/path/or/other/my.file; \
then \
echo File Exists \
else \
echo Doesn't Exist \
fi

so let that be a lesson to ye.

dPds works !

This was posted on the renderman forums, so i'm pretty sure its common knowledge.
Basically if you are in need of a consistent local basis transform for the usual stuff like normal maps and mapped anisotropic direction, you'll need a reliable tangent vector, sometimes referred to (by me, mainly) as dPds.

The RSL code is as follows:


void dPdst( float ss,tt; point PP;
output varying vector dPds;
output varying vector dPdt; )
{
float Dus = Du(ss);
float Dut = Du(tt);
float Dvs = Dv(ss);
float Dvt = Dv(tt);
vector DuP = Du(PP);
vector DvP = Dv(PP);
float det = (Dus*Dvt-Dvs*Dut);
dPds = (DuP*Dvt - DvP*Dut) / det;
dPdt = (DvP*Dus - DuP*Dvs) / det;
}


9 things i hate about my iPhone

These are the 10 9 most irritating things about my iPhone that stop at 'quite good' where it could have been truly great.

1 - Battery Life : shocking.
2 - Its expensive. Very very expensive.
3 - It can't bluetooth photos to non-iPhones. Basic stuff, and it can't even do that.
4 - You can't use your own sound files for ring tones. You have to buy them
5 - You can't copy files, such as pdfs, to it for offline reading. That's rubbish!!
5 - The calendar can't repeat on useful intervals such as 'last working day of month' or 'first tuesday in month'. Basic, basic, *basic* functionality that it can't even do.
6 - The pin number screen-lock is so irritating. If I want my data secure, I have to keep unlocking it all the time, even with the least annoying unlock option. Its so annoying when you are in a hurry.
7 - Touch sensitive keypad and predictive text frequently cause typos. The more of a hurry you are in, the worse it gets since it is so over-sensitive to mis-keying (or not adding the letter even when you hit a button with your finger).
8 - You can't write your own software for it unless you have a mac. Why should that be? Its not even running OSX !
9 - The sim card slot is like a chinese maths puzzle.


And that ladies and gentlemen, is why I think Apple are such smug bastards.

handy shell alias

pf is a wildcard running task search :
alias pf "ps -ef | grep \!* | egrep -v 'grep |bin/pf'"


rmane set prman env :
alias rmane 'setenv RMANTREE /opt/pixar/RenderManProServer-\!:1 ; setenv PRMANVER \!:1_64 ; setenv PRMAN_SHADER_PATH ${BUILDROOT}prman-${PRMANVER}/shaders ; setenv PRMAN_DSO_PATH ${BUILDROOT}prman-${PRMANVER} ; echo RMANTREE set to $RMANTREE ; echo PRMANVER set to $PRMANVER ; echo PRMAN_SHADER_PATH set to $PRMAN_SHADER_PATH ; echo PRMAN_DSO_PATH set to $PRMAN_DSO_PATH '


rmanv queries prman env :
alias rmanv 'env | grep RMAN'


ribgrep :
alias ribgrep '~/ribgrepext \!*'


where ribgrepext is a bash script like so:

ribgrepext :
#!/bin/bash

if [ ! $# == 2 ]; then
$RMANTREE/bin/catrib $1 | grep -o -e '[^\"]*\.exr\|[^\"]*\.dsh\|[^\"]*\.shd\|[^\"]*\.ptc\|[^\"]*\.optc\|[^\"]*\.bkm\|[^\"]*\.tex\|[^\"]*\.unk' | sort | uniq
else
$RMANTREE/bin/catrib $1 | grep -o -e [^\"]*\.${2} | sort | uniq
fi


Shader DSO Global Storage

RixStorage* storage = rslContext->GetGlobalStorage();

This is global across all plugins ! So two plugins using the same string key in

void* myglobal = storage->Get( *inArg );

will behave differently to one plugin. I.e. to the second of the two plugins, it will appear that the string was already encountered by this function, when in fact it was not. Use the thread storage in that case, but remember the locking to make it threadsafe has an speed penalty.

ldd for static libraries


objdump -af


Vector Basics

Reflect I about N

I - 2*(I.N)*N















Project d onto the plane that N is normal to

d - (N.d) * N

help for shake scripters

This is so cool, I can't beleive I didn't know about it before

shake -help |& more

Previously I only knew that shake -help will output on stderr, not stdout so (under tcsh) I can pipe that to a file using >& (instead of > which pipes stdout to a file ) but then I have to open the file and read it, whereas all I want to do is | more

So the | pipe symbol works on stdout and |& is the same but stderr, which is where shake -help outputs to.

gnu plot tricks

in case anyone else needs to, the way to store a variable and use that instead of directly using equations is;

plot f(x) = cos(x/180.0*3.141592), f(x)**2 * (3.0 - (2.0 * f(x)) ), f(x)**2

where "f(x)" is the name of the 'variable'

4 digit padding in mel (using python)

string $padframe = python( "'%04d' % " + $newFnum );

symbol table utility

    nm -A -C dof.so | grep globalThreadCount

This is useful to track the source of an error message related to a mangled sym table name,
eg.
    D02001 {WARNING} TgPathDLOpen: Bad DSO found: dof.so (undefined symbol: _ZN3Imf17globalThreadCountEv)


    man nm
   
will give the manual page


    ldd mydso.so

will list the dependencies a dso has. Which is also useful

Cross Product, Right Hand Rule

1stFinger x 2ndFinger = Result

The order in the equation has to match the spatial order in the picture. 1st Finger comes first when reading equation left to right.

reducing errors

Put this in the rib. Sweet!

       ErrorHandler "printonce"

stl in gnu / gcc

Listening to : Werefrogs, "Forest of Doves"


So i wanted to use the stl pair and couldn't remember how to include it in ( #include < pair > didn't compile ) and I found out that in gcc334 pair is deprecated, but still available via #include < bits/stl_pair.h >


then use it with the std namespace as per usual.


The other one that's slightly different is hash_map, which is part of the extension.
Include it like this;


#include < ext/hash_map >


and use it with the __gnu_cxx namespace
like this


__gnu_cxx::hash_map < int,double > X;


voila!


(oh and by the way these should be automatically included by gcc, no need for -I in your makefile or anything)

Rib attributes in a Slim Template

Listening to : Blue Jam

Here's the syntax for adding a Rib attribute to your Slim Template, so that it appears as a built in control.

       collection void RIB {
           state closed
           ribattribute string "shade:transmissionhitmode" {
               label "shadeTransmissionHitMode"
               default "primitive"
               subtype selector
               range { "primitive" "primitive"
                       "shader" "shader"
               }
           }
       }

Rotation Matrix from Axis Vectors

Listening to : The Decemberists (various)

This is an oldie but a goodie, and I haven't posted it anywhere else, so here it is before I forget.

If you want to make a rotation matrix ( 4x4 ) so that you can make objects orient the same way as, say for example a normal and a tangent on a surface, so long as you have 2 of 3 of the desitnation "axes" then you can build a matrix directly from copying the values of those vectors. (if you do only have two, you get the third from doing a cross product).

I'm using row major notation here - so anyone who needs this column major, just transpose it.
I'm also assuming that the space of the sought rotation matrix is the same space that the vectors are in. (e.g. world space)
Lastly we're assuming that the normal and tangent and binormal are all normalized.

So, you have a normal and a tangent in (say) world space and you want a rotation matrix so you can apply the same orientation to other objects. This is the same as making a new coordinate system and transforming them into it without any rotation at all in that new coordinate space.

It helps to think of normal and tangent, and the 3rd vector called the 'binormal' (which is the cross product of the other two) as the axes of a space into which we're transforming.
It can be any order really, so lets pick :-
Normal = +Y axis
Tangent = +X axis
Binormal = +Z axis

So we have (for example)
+X vector is ( 0.670014, 0.541446, 0.507856 )
+Y vector is ( -0.609193 0.791979 -0.0406534 )
+Z vector is ( -0.424223 -0.282144 0.860482 )

So our 4x4 matrix will be
| Xx Xy Xz 0 |
| Yx Yy Yz 0 |
| Zx Zy Zz 0 |
| 0 0 0 1 |

which in our example gives us the following 4x4 rotation matrix !
| 0.670014 0.541446 0.507856 0 |
| -0.609193 0.791979 -0.0406534 0 |
| -0.424223 -0.282144 0.860482 0 |
| 0 0 0 1 |

So all that any rotation matrix consists of, is the three axes vectors that define it as a coordinate system.

To rotate a plane by -45 degress about X in MEL, you'd do this
// -45 degrees in X
xform -m 1.0 0.0 0.0 0.0 0 0.707107 -0.707107 0.0 0 0.707107 0.707107 0.0 0.0 0.0 0.0 1.0 pPlane1;

Where 0.707107 is of course 1/sqrt(2)

Accented Characters on Linux

Alt Gr + ;
e
= é

Alt Gr + ;
a
= á

Alt Gr + '
e
= ê

Alt Gr + '
a
= â

Alt Gr + #
e
= è

etc.

Faking depth in a cyc arena

Listening To : Prince, Raspberry Beret

A problem that came up during Flyboys was that we had a tiled arena floor that was 10kms square, but beyond that we had a matte painting 'cyc' aka cylindrically projected onto a bowl shaped geometry with vertical sides.
When rendering a Depth pass or secondary output, when a camera following the action got close to the bowl, then the colour information in the matte tells you there are things far away in the distance, but the depth (as in length(I) ) doesn't. The depth values decrease rapidly as you approach the sides of the cyc.

Not what we want at all.

So if its safe to assume the ground continues in a near-planar fashion in all directions out to the horizon, we can do a very simple, quite elegant cheat to work out what the depth would be, if the cyc wasn't there.

Breifly (see diagram also):

float  S  = 1;
point  P  =  transform("world", P);
point  E  =  transform("world", E);
vector I  = vtransform("world", I);

if ( ycomp(I) < 0 )  // I.e. point being shaded is closer to ground than the camera
{
 // S is some scale value we need to solve to scale I upto;
 //  I' = P'-E
 // ycomp(P') we know is always zero
 // therefore we can solve the scale for ycomp and use it to scale I (similar triangles)
 //  P' = E + I*S;
 // so;
 //  ycomp(E) + ( ycomp(I) * S ) = 0;

 S = ( -ycomp(E) / ycomp(I) );

 I = vtransform("world", "current", I*S);

 // Voila, our new fake (clamped) depth is;
 depth = min( length(I), 65535 );
}
else
{
 // E.g. plane is upside down, etc.
 // Just Set depth to Max ( 65535, which is the largest value our EXR's can store )
 depth = 65535;
}

occlusion, subset and the minus sign

Listening To : 'The Now Show', Radio4.

I didn't know that you could negate subsets the same way as you can for light categories, but it totally works. You need to assign the group via rib attributes, to the geometry.
e.g.
Attribute "grouping" "string membership" ["noRflOccl"]

Cool.

Current Project Path via MEL

Listening To : "Its Art, Dad!", The Clientele

Here's a handy tip. How do you get the current project path from MEL ?

string $ws = `workspace -q -rd`;

Voila!

Under the hood

Listening To : 'Sensitive Artist' by King Missile.

To paraphrase an email from a work colleague..

Pixel samples have no effect on when and where the shader gets executed. The shading rate and other dicing settings determine how the geometry is tessellated into micropolygons. Then the shader is executed at each vertex on the micropolygon. The pixel samples then just determine how this pre-shaded micropolygonal geometry is sampled for each pixel.

Not only do pixel samples have nothing to do with shading, they also have nothing to do with raytracing. Shooting rays into a cone an angle of 0 will always hit the same spot regardless of samples - the pixar docs appear incorrect about this, based on experimentation. Also, it's not necessary to specify the cone angle at all if using the simpler environment() and trace() routines. I believe renderman will compute the optimal cone angle automatically based on shading rate and surface derivatives. Although in tests, geometric aliasing still occurred with trace, and only making a coneangle based on the change in incident angle across a micropolygon with gather seemed to remove it.

Watch those param orderings!

Listening To : Shooby Taylor, the human horn.

Refract and Faceforward have I and N paramaters specified in different orders! How many times has that caught me out ?!?!?

     vector refract( vector I, N; float eta )
     vector faceforward( vector N, I )

Also clamp and mix/smoothstep have different orderings.

     type clamp( type a, min, max )
vs
     type mix( type x, y; float alpha )
     type smoothstep( type min, max, value )


MEL message attributes

Listening To : Bill Bailey , Jimmy Cliff

MEL today.
Just found out about message attributes and connecting things together.
connectAttr and listConnections. Neato.

Also, thought for today is that in Games, the artists always seem obsessed with the characters animation, run cycles etc. and don't really care about lights or cameras too much. In post, lights and cameras are so much more important (duh). Yeah, well, not a fabulous insight - but was just thinking about the differences earlier today.

DNeg Free beer and pizza tonight, and my contracts just been renewed. Yippee!

the grids

Listening to: Nothing, i've lost my iPod charger cable

So to wrap up my previous post about the mysterious grid lines..

It turns out that in multiplying my derivatives used in the texture map lookup in order to blur the map, the usually unnoticeable discontinuities in derivatives around the shading grid boundaries had been magnified (by the multiplication).

I was doing this:-

duds * 25 = ...
dvds * 25 = 0.03 // for arguments sake, it's there or thereabouts
dudt * 25 = ... // similar order of result for the others
dvdt * 25 = ...

but sometimes the derivatives were very different around shading grid boundaries, so the tiny aliasing that is normally unnoticed, was being amplified and appeared as a grid.

The solution was to add a number to the derivatives to get roughly the 0.03 I wanted (so not 25, something much smaller instead). That way the aliasing would stay small, but I would still get the blurring I wanted.

So that's that.
Hurray!

brickmaps, pointclouds.

Listening to : Fiery Furnaces 'EP' and The Crimea's new CD 'Tragedy Rocks'

So it looks like the best thing to do for my st bake is to have "interpolate" set to 1 in bake3d to get micropolygon midpoints (from four sample points) instead of absolute points and also to avoid duplicate points being written out at the boundaries. Then, in texture3d, use "lerp" set to 1 so that it looks up from the two nearest mip levels and interpolates between them.

Curiously, I found that baking at 1200x1200 shading rate 1 gave one quarter of the points (2million instead of 8million) and was 80Mb instead of 345Mb, but had much much better results and a more consistent point cloud compared with previously at PAL res, same shading rate, when it had been patchy..
Maybe this was due to adding interpolate==1 as mentioned above ?? If so; way-hey! Interpolate is cooool!

(Note to the newbies - generally to increase point cloud density you lower the shading rate and/or increase the render res, which is what made this result so odd).

In the beauty render, I was seeing dark grid lines aligned with what looked like the shading grid. At first I guessed somewhere I made the classic misktake of doing texture lookups or area functions inside a branch instruction that uses a varying variable instead of a uniform one. This is such a common mistake when you're used to writing real-time code with efficiency in mind. However, after fixing these in the shader, it's still there, so it is probably something else..

Other problems I have at the moment are;
- the baked average ray direction looks grossly incorrect. Not really a problem but would be nice to get to the bottom of the cause.
- i'm getting some squares flickering.
- I still don't know why my brickmap is okay down to level 7, but has mostly black beyond that. It means I have to set maxdepth to 7 before I render the beauty pass... kind of a pain.

Cheap global illumination

Listening to: Gustavo Santaolalla "Motorcycle Diaries", and MYLO "Destroy Rock N Roll"

So I had this problem, how do you do efficient diffuse lightspill (sometimes referred to as global illumination) when the light source is an area light that's playing an image sequence that updates every frame.

Baking illuminance or using 'indirectdiffuse' in a lightshader might seem like the way to go, except they are both really slow. What I need is a solution that will be quick to render when it comes to the beauty pass. It doesn't matter so much if the setup is slow (in the same ballpark as occlusion is slow anyway).

So I settled on a shader that shoots rays from geometry in the scene towards the image sequence light source, and averages the texture coordinates that get hit, then bakes this to a pointcloud (along with a scalar ratio of hits to misses and an average direction vector). This way, so long as the scene is static, we can bake the spatial relationship of geometry wrt the light source and then use these baked texture coordinates as a texture lookup that's cheap to do. We can change our image sequence each frame, and the texture coordinate lookup is still valid.

They are a bit like shadowmaps though, you need one per light source, and if the scene is non-static, then you need to bake once per frame that's got geometry in a new position. But for a static scene, they are a winner because they are view independent.

So why didn't they work at first when I used the bake3d and texture3d calls?

Well, if you have a cloud of points, then you need to know what space they are stored in and what space is used in the retrieval.

So the answer, and the tip for today, is that bake3d and texture3d by default will transform P and N that you give it into world space before doing the lookup. That means you don't usually have to care about what space the P and N are in that you pass to it. (Not counting Pref etc, of course. But my scene doesn't have any deformations, so no problem).

First post!

Listening to : CD of Mighty Boosh radio show.

Wow, first post! I guess I wanted to set this blog up ages ago. In fact, back in 2003 when I first started using renderman in film vfx for my living, I wanted to keep a log of all the things I had to learn to re-train from being a 3D Programmer in the games industry. That opportunity to chronicle the initial learning curve involved has probably passed, rats, but hey-ho I can still post things as I learn them....