且构网

分享程序员开发的那些事...
且构网 - 分享程序员编程开发的那些事

GLKit遮罩或混合jpegs中无2种纹理

更新时间:2023-01-27 10:06:53

GLKit是一个框架,具有四个主要组件,可以一起使用或分别使用:视图/视图控制器,数学库,纹理加载器和效果。我从您的问题假设,您担心的问题 GLKBaseEffect

GLKit is a framework with four major components that may be used together or separately: view/view controller, math library, texture loader, and effects. I presume from your question that the issue your having concerns GLKBaseEffect.

一个人可能会说 GLKBaseEffect 的目的是简化日常工作。它主要是对OpenGL ES 1.1固定功能管道的替代,并完成了一些常见的技巧。

One might say GLKBaseEffect's purpose in life is to make routine things easy. It's primarily a replacement for the OpenGL ES 1.1 fixed-function pipeline and some of the common tricks done with it.

您追求的不是常规的固定-函数流水线任务,因此要做好,就意味着要超越 GLKBaseEffect 的基本功能,并编写自己的着色器。那是个坏消息。好消息是,一旦您进入了着色器世界,这将非常容易。另一个好消息是,您可以期望获得出色的性能-您可以在GPU上执行此操作,而不是使用CPU来进行CoreGraphics混合。

What you're after isn't a routine fixed-function pipeline task, so to do it well means stepping beyond the basic functionality of GLKBaseEffect and writing your own shaders. That's the bad news. The good news is that once you're into the shader world, this is pretty easy. The other good news is that you can expect great performance -- instead of using the CPU to do CoreGraphics blending, you can do it on the GPU.

部分,如果您不熟悉OpenGL ES 2.0(或3.0)着色器,请使用自定义代码替换 GLKBaseEffect 提供的客户端OpenGLES代码和顶点着色器,相同。不过,这里有很多示例代码和教程。 (由于您一直在关注Ray Wenderlich的网站,因此我确定您会找到不错

The "hard" part, if you're unfamiliar with OpenGL ES 2.0 (or 3.0) shaders, is replacing the client OpenGL ES code and vertex shader provided by GLKBaseEffect with with custom code that does the same. There's lots of example code and tutorials for this, out there, though. (Since you've been following Ray Wenderlich's site, I'm sure you'll find some good ones there.) The main things you need to do here are:


  1. 从坐标转换顶点 glDraw 命令所使用的空间到剪辑空间。对于2D东西,这非常容易,因为您可以使用与 GLKBaseEffect 变换相同的矩阵设置属性。

  2. 加载两个纹理( GLKTextureLoader 可以在这里帮助很多)并将它们绑定到两个纹理单元。

  3. 在客户代码中,将每个顶点与一些纹理坐标关联。 (您可能已经在这样做了。)

  4. 在顶点着色器中,从 uniform 输入中获取纹理坐标并将其传递到变化的输出。 (除非出于某种原因要变换坐标,否则不必执行其他任何操作。)

  1. Transform vertices from the coordinate space you're using for your glDraw commands into clip space. For 2D stuff this is pretty easy, since you can use the same matrix setup you were doing for GLKBaseEffect's transform property.
  2. Load your two textures (GLKTextureLoader can help you a lot here) and bind them to two texture units.
  3. In your client code, associate each vertex with some texture coordinates. (You're probably doing this already.)
  4. In your vertex shader, take the texture coordinates from a uniform input and pass them to a varying output. (You don't have to do anything else unless you want to transform the coordinates for some reason.)

真正实现您想要的效果的部分非常容易,而且全部都在片段着色器中:

After all that setup, the part that actually accomplishes the effect you're after is pretty easy, and it's all in your fragment shader:

// texture units bound in client code
uniform sampler2D colorTexture;
uniform sampler2D maskTexture;

// texture coordinate passed in from vertex shader
// (and automatically interpolated across the face of the polygon)
varying mediump vec2 texCoord;

void main() {
    // read a color from the color texture
    lowp vec3 color = texture2D(colorTexture, texCoord).rgb;

    // read a gray level from the mask texture
    // (the mask is grayscale, so we only need one component
    //  of its color because the others are the same)
    lowp float maskLevel = texture2D(maskTexture, texCoord).r;

    // use the gray level from the mask as the alpha value in the output color
    gl_FragColor = vec4(color, maskLevel);
}

此外,由于您提到以24 fps的速度执行此操作,这表明您在使用视频。在这种情况下,您可以删掉中间人-视频帧已经在GPU上处理了,将它们放入CPU的CGImages中只会减慢您的速度。看看 CVOpenGLESTexture 类和 GLCameraRipple 示例代码,可帮助您将视频转换为纹理。

Also, since you mention doing this at 24 fps, it suggests you're working with video. In that case, you can cut out the middleman -- video frames are already being handled on the GPU, and putting them in CGImages on the CPU just slows you down. Take a look at the CVOpenGLESTexture class and the GLCameraRipple sample code for help getting video into a texture.