Devlog #3 - Scripting API for my web avatar chat (webrtc)

Devlog #3 – Scripting API for my web avatar chat (webrtc)

Here I give an example coding along inside of my webrtc driven creative platform. If you are looking for interactive coding or follow along projects. Welcome aboard – at your request I will post the follow up video demonstrating this script running.

Transcription of the Video:

Introduction to Kynto Update Release Process (CI/CD)

0:02 today I’m here to release an update for. Kinto I do have my laptop here this is

0:08 me streaming myself into Kinto why am i

0:10 showing you this because I’m about to

0:12 show you what’s involved in releasing an

0:14 update for kto I’m going to show you the

0:16 the build pipelines the deployment

0:19 processes and continuous Improvement

0:21 that I’ve implemented at kto to make

0:24 working on this thing easier I have just

0:25 spent the last hour an hour and a half

0:28 committing all my changes from last uh

0:31 30 OD days of development has occurred

0:33 solely on the development machine right

0:36 here you can see that last update I

0:38 pushed out was in April 8th so maybe it

0:40 wasn’t as long as I

0:42 thought but that’s been a while so we

0:45 build the client Client Build Pipeline Overview (Jenkins)

0:50 first so we look at this these are all

0:53 the commits that are being built

0:56 today um this is the process that is

0:58 occurring this is a cust build script so

1:01 it builds on my build server which is 51

1:04 here creates directories and populates

1:06 our release folder switches out the

1:09 environment files with uh the product

1:11 environment file and creates an archive

1:13 and once it makes the archive it uploads

1:15 it to AWS so it’s fully built there and

1:17 just supplies it straight to one of my

1:20 servers the client is typically like one

1:22 of the faster parts of the project to to

1:25 build

1:30 so nice to finally be back in see green

1:34 check marks hopefully the PO is okay to

1:37 AWS and it did so make the Avatar’s face



today i’m here to release an update for Kynto. i do have my laptop here this is me streaming myself into Kynto why am i showing you this because i’m about to show you what’s involved in releasing an update for Kynto i’m going to show you the build pipelines the deployment processes and continuous Improvement that i’ve implemented at Kynto to make working on this thing easier i have just spent the last hour an hour and a half committing all my changes from last 30 OD days of development has occurred solely on the development machine right here you can see that last update i pushed out was in April 8th so maybe it wasn’t as long as i thought but that’s been a while so we build the client Client Build Pipeline Overview (Jenkins) first so we look at this these are all the commits that are being built today um this is the process that is occurring this is a cust build script so it builds on my build server which is 51 here creates directories and populates our release folder switches out the environment files with the product environment file and creates an archive and once it makes the archive it uploads it to AWS so it’s fully built there and just supplies it straight to one of my servers the client is typically like one of the faster parts of the project to to build so nice to finally be back in see green check marks hopefully the PO is okay to AWS and it did so make the Avatar’s face”

1:41 map directly to the Avatar’s face so Face Mapping Feature Implementation (TensorflowJS, webrtc)

1:43 what I’m talking about here is facial

1:46 mapping and we have this in Kinto uh

2:03 so that’s uh the face mapping is this it

2:06 lit takes you the Avatar and allows your

2:08 face to be dynamically mapped so really

2:11 cool that’ll be going out it will be

2:13 disabled initially it’ll just be

2:14 something I’ll be experimenting with

2:16 making sure it works properly this is

2:18 going to require a whole episode on its

2:20 own like how it actually implemented

2:22 face mapping users face to the pixel

2:24 Avatar like it requires more technology

2:27 than you expect like this is using

2:28 machine learning inos intens

2:30 4js um that’s just to start so we’ll

2:35 pick that back up at another

2:37 time Implement Dynamic capture scene

2:40 media stream height and width only hide

2:42 Avatar video if it is not a face Ma on

2:45 chart

2:50 uh tidy up Bas extractor load model

2:54 method unload YouTube If room updated

2:56 events contain show Falls no longer

2:59 manually move Avatar mass is an ISO

3:02 spray big bug preventing tile from

3:04 restoring uh opacity if the alpha was

3:08 Zero move render badges to draw function

3:10 in Avatar as it was been triggered

3:12 incorrectly um Implement to destroy

3:15 video tracks which fixes a bug where

3:17 destroying an Avatar video would destroy

3:19 the media stream uh Implement capture Dynamic Capture Scene and Media Stream (Canvas, webrtc)

3:22 scene which lets users without media

3:25 devices also stream using the Avatar in

3:28 game and also like a slice of the World

3:30 Behind them and that stream in the

3:32 Stream panel just like you’re seeing now

3:33 with my video here uh so that’s really Implementing Audio Mode for Webcam (HTML5 MediaStream)

3:40 cool Implement audio mode for webcam so

3:43 I just showed you the face map feature

3:45 previously you could only just well in

3:47 development I could only stream video

3:49 but I combined audio and video media

3:53 stream um by just adding tracks to the a

3:56 new media stream combining a video and

3:58 audio into new stream actually

4:00 transmitting that we’ll see how that

4:02 behaves directly so I’m going to go

4:03 ahead and release the the game part now

4:05 the world or game it’s called game it’s

4:08 actually the UI so I’m going have to re

4:10 tight of that build

4:13 that probably going to be a lot more UI (Game) Build Updates (React & React Bootstrap)

4:15 changes

4:18 here yeah so 24 changes

4:30 Implement clean messaging sidebar are

4:32 you let it take whole width containers

4:34 set body background to Black decrease

4:37 font size of Chat bar localize and set a

4:40 more readable shared message oh okay so

4:44 that’s if you like to click to share

4:46 room uh it gives you a little popup if

4:49 you’re on OS that supports the native OS

4:52 share if you don’t it just copies the

4:55 room URL to your clipboard and then

4:56 notifies you notifies you vers with an

5:00 hey you’ve got this in your clipboard go

5:02 ahead and paste

5:03 it interesting happened that this part

5:06 of the video while I was editing the

5:08 project actually became corrupted and

5:11 there was no way to recover it

5:12 fortunately I was able to make a render

5:14 before that happened so there’s a ton of Peer-to-Peer Logic and Platform Communication (WebRTC)

5:19 updates included in this specifically

5:21 focused around video chat and streaming

5:24 the majority of the peer-to-peer logic

5:26 is contained within Avatar and it emits

5:30 all sorts of events that talks to

5:32 various parts the C platform and

5:33 application including the native layers

5:36 the desktop layer obviously the web

5:38 layer iframe layer and I believe that’s

5:41 it right

5:43 now so for the last 30 days or so I have

5:46 been building up features and Bug fixes

5:51 for my platform

5:53 Kinto that are not released yet and this

5:55 is actually quite a problem especially

5:57 for Jamie and a over there she wants to

5:59 she enjoys desktop streaming and the

6:01 easiest way to do that is with Kino like

6:04 it’s just easy she could just grab her

6:05 phone she can log in her computer go to

6:07 the same room uh share her desktop in

6:09 the computer and share her uh video and

6:12 audio on the phone boom you’re on it

6:15 you’re on you’re on K to you’re on

6:16 YouTube You’re live and it’s fantastic

6:19 like that but that’s not good if she’s

6:22 like loading my local IP on the network

6:24 and I’m trying to do updates or you know

6:27 that’s not going to work so we’re going

6:28 to get those we’re going to get that

6:29 figured out right Importance of Source Control and Conclusion (Stay tuned!)

6:32 now I’m going to talk a little bit about

6:34 what AIT is sure a lot of you know what

6:36 AIT is but um Source control is very

6:39 important in uh development I’d say it’s

6:43 almost it’s impossible to work without

6:45 um with production facing systems

6:48 like even just the editors we use to

6:50 code when you save a file they can mess

6:53 up and interject character you got have

6:55 a cat walk your keyboard you could

6:57 accidentally hit a key and you don’t

6:59 know that and you scroll down to the

7:01 piece of code you’re working on two

7:02 hours later you go to release that put

7:05 it on production or whatever you don’t

7:07 have any tested steps you’re going to

7:08 run into some pain

Leave a Reply

Your email address will not be published. Required fields are marked *