sg13

From: Roger Orr <rogero_at_[hidden]> · Date: Tue, 23 Jul 2019 19:54:23 +0000

--
---+++ SG13 After Lunch
There were 18 attendees present
We began with a discussion of P1108R2: web_view
---++++ Discussion
(Presenting) This is an updated paper since reviewing in LEWGI (R3 to come in post-meeting mailing)
There are multiple IO proposals, this is a proposal that tries to ‘bite off all of the enchilada all at once’
It is complimentary to the group level up approaches, and there is a lot of value in those proposals
We need to think really hard about user interaction, especially on modern devices – desktop & mobile
Also need to support auxillary technologogies, not just rendering text but also things like screen readers, accessibility, audio, video, etc
The question is: How should a modern C++ application interact with this huge ecosystem?
I think the answer should be to outsource as much as possible to other existing groups that already have existing solutions (HTML, Javascript, etc)
The benefit of the proposal being short …
Web technologies are complicated; there are many implementations: WebKit, Gecko, Microsofts implementation and Chromium
Links in the paper to the different APIs
Web is a complicated space; you need the ability to display some window and then ask the system to display some content
I think the important thing here is is that you have to think about this as a system service
There is not a standardized interface yet, but there is a common set across these APIs
I do want to mention quickly – people have asked – how does this compare to standardize a native API that these technologies covers?
If you look at Qt, they have ~1600 classes and ~13000 functions, it is not easy
QML/QtQuick is another alternative but 3 thoughts on this:
1. There is 1 version for mobile 1 for desktop
Q: QML is a completely different level of abstraction; we are talking more low level than that
A: I am attempting to motivate why we are at such a high level
There were very serious suggestions saying we should take one of the existing codebases / technologes and use this as starting point. However, as ISO we can’t really just point at Qt and say do what they do
This would inevitably leave to a ton of arguments
Q: Should be sent to SG16 for Unicode reviewing
Commenting on the long list of standards: Legislating this list should be under SG13’s perview; and I think this is a bad idea
These standards don’t evolve together; you should just pop a web_view to whatever you want
You are effectively saying I need to go to the vendor
Reply: I went to caniuse.com, and this is where this list came from
Comments: We can’t surface all of this stuff, such as geolocation – this requires user consent; there is a ton of complexity in this stuff
You are also legislating legacy technology to be maintained
I agree that this should become a quality of implementation in terms of what you support, however, I would like to see if there is a bare minimum
Comment: Graphics, audio at some point to be supported; native approaches / techniques might be procluded by your approach
Is this designed to be a closed list?
A: No, I am not proposing this to cut off certain technologies
Everyone knows what this list does
Q: Do you know what that is? I don’t. And how would you standardise this?
Some of this stuff v few ppl understand b/c it is too new or now enough implementations
So we seem to be recommending standardizing existing practice
QtWebView / MacOS WebKit do something similar to this
Q: My question is there any examples similar to what we are trying to do here?
A: Not to my knowledge
Sciter, they have HTML5 and CSS and you can do C++ applications with this; this 5 MB; it is propietary; UltraLight is another example of something similar
( back to demoing the interface / use case of the interface - 1st example - “Time has passed” demo)
There is a custom URI scheme registered
WxWidgets provides this ability which Qt does not
The stuff happening on screen is not being run by Javascript, it is being injected by the C++ code
Q: How does this work?
I will show you
( back to code behind demo )
Q: This is pretty worrying as an API. One of the most common uses of webview is a payment page.
The problem is that the application does not own the payment part. There are native ways to do payments, lets ignore that
You the app, aren't the same owner as the page you have opened
This allows you open a webpage and then inject code into a ‘page’ that you do not own
A: Yes I saw a way to do that with the frameworks, which probably creates a problems
There are ways to do this, i.e. HTTPRequest, and then the webpage can (localhost) agree to let this work
Good thing to note
Q: Depending on how the app is setup and presents it self as a browser wrapper vs an engine on the platform are two different cases.
*Summary:* Javascript pages with injection into pages it does not own has security implications.
Q: Is there any kind of synchronization with script on the page vs navigating to another page
When you ask it to display this URI, you do have to wait until the page is done
Q: I am in a while loop, window wasnt closed, I come to run script – I attempt to run it. How can I be certain I have the correct page?
This is a good point.
Q: Looks like you used a custom URI Scheme. Can you explain that?
The scheme only works within the app.
Q: OK. What happens if you do display from URI back to back from multiple things?
Webview maps to one logical window.
Q: And you can create multiple webviews?
Yes. The way I envision this working, if there are system limitations preventing this, the api returns error codes so it could fail
Q: On the scheme handler, this acts synchonously, if I want to do blocking calls – is there a design choice for not returning a future or suspending this? Or even forwarding to an existing handler to do some wrapping or something like that?
If you want to get a request, and cause the whole page to navigate somewhere else, you would return false and then call a displacement URI
I don’t have anyway providing a wrapper of an HTTP implementation-defined
Note that there is not parameter that gives you any auxilary data here.
Comment: When you register a URI scheme handler, aka I pop open www.google.com – and then the URI scheme can use that
I think so
*Summary:* Can the webpage make requests to that URI Scheme? The answer is yes.
Comment: I was worried about was how to C++ talk to the webpage, and vice versa. This addresses that.
( back to demoing code )
Notes that due to asychronous nature this needs to go through SG1
Q: What does irrevocably unusable?
That was my best guess at how to word a window closing
Q: Just concerned about translating to starndardese
It is supposed to mean the user cannot get to it anymore, it was my best guess at it – but I am hapy to take suggestions on that
I have a second demo
Using SlickGrid and UIKit to look really nice
The app is very similar to the other one
Q: So the slider is C++ injected?
No, but the data in the table is
So that is the interface – I think this is a nice area for standardization
It lets user take advantage of the vibrant ecosystem, write modern C++ looking apps and it allows us to reference existing standards that takes advantages of existing technologies for rendering, etc.
Q: Is there a link to a github where I can play with this?
A: It is hidden in a paragraph. WebView link right after the code.
Or: [[ https://github.com/hfinkel/web_view ]]
Comment: my company supports this proposal and if SG13 is going to prioritize a graphics proposal, this is the one we think that should be prioritized
There are issues around asynchronous behaviour (i.e. win32). The way callbacks are handled, there is more play there. The way the underlying implementations work, there could be nice solutions here.
Q: Have you considering an interface that exposes that underlying implementation?
You can provide native handles sometimes. Native handle to what? In this case, I could imagine a native handle to the window, etc.
Comment: Generally, create a struct that contains all the handles in one place.
Comment: On the scheme handler, we definitely want to be able fetch thing over the network. We need some form of asychronisty – but not necessarily going to the underlying implemetation.
SG1 will be could to get feedback on executors interop
Comment: Or coroutines
We need to figure out on all systems that are relevant, if it is more reasonable for allowing the handler to block and allow multiple vs not blocking and having more aschronousity.
Comment: I also agree that native handle is important, you might be overthinking it – but really all you need is an entry point. The frame would probably suffice.
When ppl proposed adding native handles, comments were made if there always was a unique native handle – but yes we want to do the thing that is most useful
Q: Would this run full screen?
Great question. Right now, the prototype runs full screen (or techincally maximized). On some platforms not such thing as not full screen. You might want to do something like set a minimum size.
Are you worried about being too big? Too small? Whats the concern?
Comment: Flexibilty
Q: You have run_script? An evolution of that is loading and parsing being split.
I could provide that interface, but it probably doesnt change much.
Comment: But it is still a nicer implementation. I see this as the main interaction with C++ with the web part
Q: Have you thought about make the interface more sophisticated in terms of types?
Q: More type safety
Imagine a future where WASM (Web Assembly) is ubiquitous, supporting multiple types of scripts – if you had some type of WASM blob to run, do we need another interface?
Comment: First we need UFT-16. But ignore. WASM is 87% supported by devices. WASM is neither Web, nor assembly.
Is very similar to ELF-binary in terms of idea.
It would be perfectly reasonable to have an API that supports something other than Javascript
Chair: 3:00 PM, would like to wrap up this discussion soon.
Comment: You define concepts, never use
Comment: The name is a bit confusing, web_view – similar to string_view
Comment: This is a term of art, lets leave it to LEWGI
Q: Are any other polls requested?
Q: Are other graphics/audio initiatives subsumed by this?
Q: If you open this screen, is there some way to solve this using buffers?
Comment: Bank website – I don’t want someone screenshotting my website
Comment: We aren't building sandbox
__POLL: We encourage further work on this paper.__
|SF|F|N|A|SA|
|3|7|5|0|0|
__POLL: P1108 should co-exist with SG13 graphics proposals__
|SF|F|N|A|SA|
|7|3|5|0|0|
__POLL: P1108 should co-exist with SG13 audio proposals__
|SF|F|N|A|SA|
|8|4|3|0|0|
---+ P0267R9: A Proposal to Add 2D Graphics Rendering and Display to C++
---++++ Discussion
Background
Is a paper in response to request for graphics support, SG13 was convened, paper is on r9 (12th or 13th revision though), very mature paper. Essentially, it is a drawing API. Not trying to emulate GPU programming, just trying to plot, draw text. Not a UI toolkit.
Q: So this is similar to Turbo Pascal? Access to pixels on the screen
Do not have access to pixels on the screen …
A front end, what you would expect, all the standard operations since PostScript. Backend requirements are stated and anyone can implement them for whatever platforms they please
Backend changes were added in r8. You could imagine Qt will have different backend from Microsoft.
Provides 5 fundamental output ops: paint, stroke, fill, mask, drawtext. Drawtext is a bit more complicated.
Guidance for reading P0267r9:
section 2.a.i text metrics: right now we don’t have a way to do, this is what this section is for
section 2.b command lists: these enables a set of commands to perform the 5 fundamental output ops
Q: Are there examples?
I wrote Asteroids
Yes https://github.com/cpp-io2d there is a full implementation here
That is the one that currently has text – pulls in Pango which is commonly used with Cairo that uses text.
So the main two additions in r9: text rendering and command lists
Reviewed by LWG at least 5 times, favorably
Rapperwil LWG evening session killed this proposal with P1200.
Text rendering has been added, now will go to LEWG and ultimately go to a TS
Q: Havent read r9, read r8 extensively, P1225 was created as a minimal subset of what would be needed to approve, can you speak to which of these have been addressed by r9?
Specifically a point by point response to what is listed in Section 2, 4 and 5.
Q: P1062 – similar, but different, list.
---
Summary of command lists. Provides callbacks to the 5 fundamental output ops.
Giving a summary / walking through details of command list (starting page 202 of P0267r9)
There are some unique use cases for “optional surface” where it is very useful
(Reviewing page 223) run_function
Q: You provide a command list, and one of the components can be a runfunc?
Yes
Q: More than one?
Yes, as many as you want.
Could be modified to the last new changed command list.
Q: Can we see examples on how to use this? To solve any example problems
There is at least one sample, where command lists are in the implementation
Q: Is this in the paper? Or Github
Github https://github.com/cpp-io2d
(Currently on the r9 branch - it will be merged soon)
It is in the draw Samples/draw_cpp/main.cpp
Q: push_back of clear - why is it named clear? In the scope of command list, I would think it clears commands.
It is named clear but it is the name of one of the methods
Is is confusing because of the ambiguity
I will rename this to something like clear_surface()
I will take this as an action item
That will go into r10
now review the main.cpp file
The implementation is not required to deal with any race conditions
Q: Is there specification on which thread it runs on? Or is that up to the backend?
Dealing with locks/mutexes,
Q: Do we have another place in the standard where we could have locks/mutexes?
I am more concerned when it says is may create a thread, vs it will create a thread.
I am am bound by what is in the standard
Comment: Async is a very bad example
I propose we talk about threading
We should wait until direction from SG1
This is a SG1 specific thing, and SG13 should wait
Comment: We can only use what is in C++17 right now.
It can refer to any paper published and be developed on top of that
Feedback from SG16, why not u8string instead of std::string – and it will as soon as it is in the standard
Q: I have a question on the overall concept on how to design this: the fundamental idea is that you take a concept of a 2d surface on which you can execute drawing primitive. The set of ops is unlimited on a surface if you can build new surface. How has the design arrived at
point …
*Question summary:* Why a closed set of member operations as opposed a open set of non-member operations (i.e. similar to <algorithm> algorithms)
This design doesn’t work with an because the back-end operations wouldn’t be known, and the back-end needs to know in order to implement them.
Q: These are the 5 for the next 20 years?
A while ago, things were very stateful which let you have much more freedom.
Q: Has this been implemented with graphics that uses hardware acceleration?
No, but I looked at DirectX to see if this would be successful, and in the process TS we will get feedback on that.
Q: I am trying to understand how invoking arbitrary functions would work in the middle of some serial operations.
The back-end would process things that need to be serial …
Q: If you had a command list: draw, draw, draw, draw … you would end up doing 3 batches of operations
Yes, that would be the way
Q: The basic commands take a graphics surface template argument
Is there any runtime polymorphic way to access ?
The backend defines certain named types, if you are working with a backend that you know, you can operate on this similar to a native handle – you could go in but you would have to manage your code
Q: On the basic commands, we have this class template that provides command like clear, fill, etc. Why are they different per surface?
They depend on the template parameters
That parameter is the back-end
Q: That command needs to be there for all surfaces
Yes
Q: Why are the methods dependent on the graphic surfaces
Graphics surfaces are the back end
Q: Why is basic commands a templates
When you create a command list of these things, they all store the same information,
It doesn’t seem clear to me why the class layout depends of that information
Each command is taking in state objects – yes we could create a list of data at the front end but the back end needs to do a bunch of process
Forcing a defined front-end avoids this backend processing
Comment: But I could choose to work on the backend specific part
Q: My question was: can I do this or not?
You can only run operations on the backend that are a part of that backend
Those commands are on the backend so that
Comment: Just to state it loudly, the consequence of this design – you can never plugin a backend at runtime.
Comment: As an author, I want to ship a binary that can work with many backends
This design forces me to be dependent on the backends
Comment: If we have a polymorphic design, we can never never change those base class designs.
Your backend doesnt need to only one technology. The backend could have fallbacks if something doesnt work, but that is up to the implementer of the backend.
If the acceleration abilities don’t happen … I believe you would lose a lot of the optimization
Comment: You can still allow that, and provide a runtime polymorphism interface as well
If prevents you from doing compile-time optimization
Comment: What I understand you can do – you can’t really do without the backend
In theory, you could do it without the backend being involed. But then all the states classes but be defined and the front end and then kicked to the backend. And that will take up memory, processing time in both the front end and back end
Comment: If your backend has the same interface of the presumed back ened, you could have back end compile switch to help
The idea of a backend template evolved from LEWG drove this design
Q: Would I ever need to use two backend within 1 program?
libc++ and libstdc++ is designed to support multiple backends, there is no use case for using two different backends within the same program.
I would imagine we could do the same thing here
Comment: If I wanted to write to an offscreen image, would I want to use the same backend
Fair point
Comment: Not sure we want to support that, but it is an example
If you use a different backend for write to file and writing to screen, so I dont why … unless you need to write a file
Just to kind of cut this off, I think it would be better if someone wrote a paper with what someone would want to see.
Q: Part of my confusion is what is backend?
basic_image_surface
Comment: Than I get even more command list types
>From your persepective, it doesnt matter  what back end you choose?
Comment: I would consider taking some of the feedback, instead of asking others to write a paper, I for one am not going to write a paper
In r7, we considered many design options – and we ended up with this templated design
Comment: GTK and KDE use case
The backend can support multiple implementations – but it is up to the implementer of the back end
Q: How does all of this affect portability?
Each implementer needs to create implementation for as many different backends want to supported
Comment: In chrono you can specify a default backend, in order to not right platform specific code.
If we look at the screen, there is no conditional code
Q: When as a user do I need to deal with back-ends
If you you want to optimize for your platform
Q: And how to I programmatically do that
Cairo, Qt, WebAssembly are all supported
Comment: The short answer is that you dont have to do anything, you just compile it
Q: I think that also addresses the concerns of writing a 3P library, however I do wonder if the kind of header based using approach potentially makes it a little unstable – if the header is your entry point includes that default, and if a later point that default changes – and then
you have linking problems. Does that sounds like a concern?
I dont think so. If you have a DLL, with two different backends, the DLL will always pull in the correct backend.
Comment: 3P libs cannot include io2d.h because you could get build time conflicts
Depending on the order of you includes, it will change the behavior of client project
Comment: Say I have another piece of code, and I want to build against a certain backend, if the 3P code #include io2d.h, I cant change it to depend on the different back end
Comment: So you are talking about changing the using statement
Comment: We are talking about changing the sharelibs
Comment: If you want to use a particular bakend, you can use using
Comment: but that means the 3P lib cant be written in terms of io2d.h
The 3P lib would be written as a header library
Comment: using is typedef
Comment: lets say you have a platform with muliple backends
Comment: you imply you make the selection by using
io2d.h will be provided by the vendor for there default backend, if you want a 3P with usings that would be differently named
Comment: so you dont see core platforms offering multiple of these. If you have a 3P library that wraps this and has 1 selection for this given platform – you potentially end of with clashing using statements
only std lib implementers are allowed to put something in the std namespace
Comment: that will work
Q: I would like to talk about BasicPathBuilder, this is a superset of std::vector. Containers are the most expensive type to standardise. I would suggest, take all of the functionality in BasicPathBuilder and create standalone free functions, and then use vector. These standalone
functions will rely on ranges and iterators?
Comment: It is a very specific container
I did not realize standardizing a container was expensive
Comment: It is very expensive. Also expensive to maintain
Could it be a container adapter
Comment: I believe a container adapter would be fine
Comment: You brought this forward before, and I believe we told you to get rid of that
Comment: You have provided us a use case that supports back-ends. You should be search for the minimal set of functionality that will reduce the chance for have a breaking change later
If you cut of basic_ it will remove the ability to use using
Q: Are you talking about what we would call PathBuilder?
Comment: He is folloiwng precedent
Q: If basic_image_surface, what do we call that with using?
Q: we could follow what random did?
that would be ok
for the random example, all of the potential algorithms are library implemented
Comment: I dont think included basic_  is a problems
Comment: I am looking for a basic operation that is blitting. That is a basic thing you can do.
We had blitting, and LEWG voted removing it
Q: Do you remember the reason?
It was deemed too complicated
We support sprites through surfaces
Comment: I think that Bryce and Titus need to get together and decide whether papers proposing container adapters
Comment: I am recommending that this paper come back with no containers or container adapters, however container adapters are better than containers
My understanding is that LEWG tries to be bound by past votes
I would rather have the LEWG tell us that they dont want container and switch on their previous vote.
Comment: This paper did not have consensus the last time it went to LEWG
This paper won't pass through LEWG if it is in the same form as it was when there was not consensus
P1200 stated that multiple NBs ...
Comment: I understand P1200, but the fact remains that there was not consensus the last time this paper went through LEWG, and there needs to be changes made in order for their to be a different result next time
[ to the room as the meeting adjourns ] Feel free to email the paper author the changes you would like to see