How to sample pixels around a pixel

Posted By: jumpman

How to sample pixels around a pixel - 09/07/17 20:11

Hello friends,

How do I sample pixels around a pixel in a pixel shader loop? (thats alot of pixels lol)

For example, I want to be able to go through every pixel of an image, find out color values around it, do some stuff, and add this stuff to the final render target. For pixel 1, i want to see what color the pixel to the left/right/up/down of it is, and then change pixel 1 of the final camera render target, then move onto pixel 2. I think I've got the render chain down, I am just working on the final effect shader.

I am looking at the Blur PP from the wiki as a starting point but have a few questions:

Code:

float4 vecSkill1;

float2 samples[12]
=	{
		-0.326212, -0.405805,
		-0.840144, -0.073580,
		-0.695914,  0.457137,
		-0.203345,  0.620716,
		 0.962340, -0.194983,
		 0.473434, -0.480026,
		 0.519456,  0.767022,
		 0.185461, -0.893124,
		 0.507431,  0.064425,
		 0.896420,  0.412458,
		-0.321940, -0.932615,
		-0.791559, -0.597705
	};
	
texture TargetMap;

sampler postTex = sampler_state
{
   texture = (TargetMap);
   MinFilter = linear;
   MagFilter = linear;
   MipFilter = linear;
   AddressU = Clamp;
   AddressV = Clamp;
};

float4 Blur_PS(float2 texCoord: TEXCOORD0) : COLOR
{
	float4 color;

	color = tex2D(postTex, texCoord);

	for (int i = 0; i < 12; i++)
	{
		color += tex2D(postTex, texCoord + vecSkill1[0] * samples[i]);
	}

	return color / 13;
}

technique tech_00
{
	pass pass_00
	{
		VertexShader = null;
		PixelShader = compile ps_2_0 Blur_PS();
	}
}

technique fallback { pass one { } }

Questions:

1. float2 Samples[12]...why does this have a list of seemingly arbitrary numbers? What are these numbers used for? I dont see a rhyme or reason to these values. I see that its multiplied by vecskill, but why?

2. How come TexCoord isnt incremented? How does the blur apply to every pixel without iterating through them?

Posted By: WretchedSid

Re: How to sample pixels around a pixel - 09/07/17 21:13

2 is because of 1. UV coordinates are expressed as [0, 1], so those are offsets applied to the texture coordinates. You can use vecSkill1 to tighten or loosen the whole thing to match it towards your actual texture size (because the sampler values aren't expressed in actual pixel offsets, you need some input into how big the texture is, or rather, what kind of factor you need to multiply it by to get to whole pixels) so it either samples closer or more spread out over the source texture.

Edit: Rereading your question 2: Are you asking how the blur is applied to every pixel? The GPU will execute the shader for every visible fragment, so each invocation only has to care about the one fragment it's invoked for.

Edit2: Yeah, I didn't realize that vecSkill1 was the blur strength... Sue me.

Posted By: jumpman

Re: How to sample pixels around a pixel - 09/08/17 01:07

thank you wretchedSid.

Now that I understand UV coordinates are actually 0,1, I can see why now the following snippet causes only half the screen to be post processed:

if (texCoord.y >= .5) // postprocess only the lower half of the image
{
...code
}

So the pixel shader , in a sense, applies its effect over every pixel, in this case every pixel of the render target/render buffer quad.

Why is it called a fragment? A pixel shader runs its code for every single pixel, are these the "fragments" that make the whole surface?

So knowing that UVs are 0,1, how would I be able to find the color of an arbitrary pixel within the texture? I want to find the rgb values of the pixel thats x=32 and y=245 for example, how would I do this? vecViewPort?

Posted By: WretchedSid

Re: How to sample pixels around a pixel - 09/08/17 04:04

Pixel and fragment are interchangeable, but I prefer the term fragment over pixel. The reason is that pixel is something that your display has, but a shader can run for "pixels" that are never visible like that on the screen by rendering into an offscreen buffer and doing post effects with it or whatever. A fragment makes more sense because it can be something that is output to the screen or an intermediate value somewhere.

To find the value of the pixel you'll need to know the size of the texture you are sampling. Let's say your texture is 256x256, then each pixel is 1/256 = 0.003906. vecViewport will most likely not contain the size unless you are doing post processing of some form.

Posted By: txesmi

Re: How to sample pixels around a pixel - 09/08/17 05:26

In addition, the 0<>1 range of texture coordinates embraces the full image size, from the beginning of the first pixel to the end of the last. Linearly interpolated and coords wrapped samples on those 0 and 1 tex coords will return the average between both pixel colors. The way to get the real color of a pixel is to get the sample from the center of the pixel. So the coords of the upper left pixel are half the size of a pixel, 0.5f / imageSize.