via GIPHY

Diffuse surface of a PlaneNode with a UIView

This is the easiest and the most effective way.

 let planeNode = SCNNode()
 planeNode.geometry = SCNPlane(width: 100, height: 100)
 planeNode.geometry?.firstMaterial?.diffuse.contents = myView

Since myView is UIView then myView can be view of a ViewController. Anything can be shown on a ViewController now can be shown on your planeNode

Last step, you set position of planeNode to wherever you want.
ARKit and SpriteKit Combination

Loading a video into SKVideoNode
Add the SKVideoNode into SKScene
Create a SCNPlane and diffuse it by the SKScene
Create a PlaneNode that its geometry is the SCNPlane
Add the PlaneNode into rootNode

A Brief Introduction to SpriteKit
So basically SpriteKit is Apple’s 2D graphics framework for games with physics simulation and event-handling support.

See more here
I will go ahead into some important parts that are particularly necessary for our task.

For every frame, the session will capture the image and tracking information into an ARFrame object, named currentFrame. This ARFrame object has a camera which holds positional information about the frame, along with a list of anchors.
These anchors are stationary tracked positions within the scene.
Both ARCamera and ARAnchor have a transform property, which is a four-dimensional matrix holding rotation, scaling and translation information. The current frame calculates ARCamera’s transform property, but you’ll adjust the translation element of the ARAnchor matrix directly.
The magic of matrices — and why they are used everywhere in 3D — is that you can create a transform matrix with rotation, scaling and translation information and multiply it by another matrix with different information. The result is then a new position in 3D space relative to an origin position
Implementation
Video
 let videoNode = SKVideoNode(fileNamed: "big_buck_bunny.mp4")
        videoNode.play()

        let skScene = SKScene(size: CGSize(width: 640, height: 480))
        skScene.addChild(videoNode)

        videoNode.position = CGPoint(x: skScene.size.width/2, y: skScene.size.height/2)
        videoNode.size = skScene.size

        let tvPlane = SCNPlane(width: 1.0, height: 0.75)
        tvPlane.firstMaterial?.diffuse.contents = skScene
        tvPlane.firstMaterial?.isDoubleSided = true

        let tvPlaneNode = SCNNode(geometry: tvPlane)

        var translation = matrix_identity_float4x4
        translation.columns.3.z = -1.0

        tvPlaneNode.simdTransform = matrix_multiply(currentFrame.camera.transform, translation)
        tvPlaneNode.eulerAngles = SCNVector3(Double.pi,0,0)

        self.sceneView.scene.rootNode.addChildNode(tvPlaneNode)
Website
 // create a web view
        let webView = UIWebView(frame: CGRect(x: 0, y: 0, width: 640, height: 480))
        let request = URLRequest(url: URL(string: "http://www.amazon.com")!)

        webView.loadRequest(request)
let plane = SCNPlane(width: 1.0, height: 0.75)
plane.firstMaterial?.diffuse.contents = webView
plane.firstMaterial?.isDoubleSided = true

let planeNode = SCNNode(geometry: plane)

var translation = matrix_identity_float4x4
translation.columns.3.z = -1.0

planeNode.simdTransform = matrix_multiply(currentFrame.camera.transform, translation)
plane.eulerAngles = SCNVector3(0,0,0)

self.sceneView.scene.rootNode.addChildNode(planeNode)

Marcus Do
A nature, universe, science, music, love lover