[DISCUSS] Reorganize Table API / SQL documentation

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

[DISCUSS] Reorganize Table API / SQL documentation

Robert Metzger
Hi,

I'm wondering whether we should make the Table API a bit more prominent in
our documentation by upgrading it from below "Libraries" to the same level
as "DataSet" and "DataStream".

This would also allow us to split it from one large page into smaller
sub-pages.

I think it would be nice to do this change as part of the documentation
writing of the new 1.3 features.

Let me know what you think.


Regards,
Robert
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Reorganize Table API / SQL documentation

Shaoxuan Wang
Hi Robert,
This sounds great to me. While I am in the middle of writing-up UDAGG doc
(FLINK-5905), I also feel it's not good to have entire tableAPI&SQL
introduction in one page.
We can move tableAPI&SQL under "application development", and split it into
small sub-topics, such as basic/UDF/UDTF/UDAGG/window aggregate, etc.
If this sounds good to you, I can help to refactor the pages.

Regards,
Shaoxuan


On Thu, May 18, 2017 at 5:01 PM, Robert Metzger <[hidden email]> wrote:

> Hi,
>
> I'm wondering whether we should make the Table API a bit more prominent in
> our documentation by upgrading it from below "Libraries" to the same level
> as "DataSet" and "DataStream".
>
> This would also allow us to split it from one large page into smaller
> sub-pages.
>
> I think it would be nice to do this change as part of the documentation
> writing of the new 1.3 features.
>
> Let me know what you think.
>
>
> Regards,
> Robert
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Reorganize Table API / SQL documentation

Till Rohrmann
I think we have a history of creating too long monolithic documentation
pages which are hard to digest. So a big +1 for splitting the Table API/SQL
documentation up into more easily digestible pieces.

Cheers,
Till

On Thu, May 18, 2017 at 12:01 PM, Shaoxuan Wang <[hidden email]> wrote:

> Hi Robert,
> This sounds great to me. While I am in the middle of writing-up UDAGG doc
> (FLINK-5905), I also feel it's not good to have entire tableAPI&SQL
> introduction in one page.
> We can move tableAPI&SQL under "application development", and split it into
> small sub-topics, such as basic/UDF/UDTF/UDAGG/window aggregate, etc.
> If this sounds good to you, I can help to refactor the pages.
>
> Regards,
> Shaoxuan
>
>
> On Thu, May 18, 2017 at 5:01 PM, Robert Metzger <[hidden email]>
> wrote:
>
> > Hi,
> >
> > I'm wondering whether we should make the Table API a bit more prominent
> in
> > our documentation by upgrading it from below "Libraries" to the same
> level
> > as "DataSet" and "DataStream".
> >
> > This would also allow us to split it from one large page into smaller
> > sub-pages.
> >
> > I think it would be nice to do this change as part of the documentation
> > writing of the new 1.3 features.
> >
> > Let me know what you think.
> >
> >
> > Regards,
> > Robert
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Reorganize Table API / SQL documentation

Ufuk Celebi-2
On Thu, May 18, 2017 at 1:52 PM, Till Rohrmann <[hidden email]> wrote:
> I think we have a history of creating too long monolithic documentation
> pages which are hard to digest. So a big +1 for splitting the Table API/SQL
> documentation up into more easily digestible pieces.

+1

Thanks for bringing it up
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Reorganize Table API / SQL documentation

Kostas Kloudas
A big +1 as well.

> On May 18, 2017, at 1:55 PM, Ufuk Celebi <[hidden email]> wrote:
>
> On Thu, May 18, 2017 at 1:52 PM, Till Rohrmann <[hidden email]> wrote:
>> I think we have a history of creating too long monolithic documentation
>> pages which are hard to digest. So a big +1 for splitting the Table API/SQL
>> documentation up into more easily digestible pieces.
>
> +1
>
> Thanks for bringing it up

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Reorganize Table API / SQL documentation

Fabian Hueske-2
Thanks for starting this discussion Robert.

I think with the next release the Table API / SQL should be moved up in the
Application Development menu.
I also though about restructuring the docs, but it won't be trivial to do
this, IMO because there are many orthogonal aspects:
- Stream/Batch
- Table/SQL
- Scala/Java

and sometimes also common concepts.
At the moment there are also many new features missing like OVER windows,
UDAGGs, retraction, StreamTableSinks, time indicator attributes, filter
pushdown, ...

I will try to sketch a new structure in a Google Doc in the next days and
share it in this thread.

Cheers, Fabian

2017-05-18 14:03 GMT+02:00 Kostas Kloudas <[hidden email]>:

> A big +1 as well.
>
> > On May 18, 2017, at 1:55 PM, Ufuk Celebi <[hidden email]> wrote:
> >
> > On Thu, May 18, 2017 at 1:52 PM, Till Rohrmann <[hidden email]>
> wrote:
> >> I think we have a history of creating too long monolithic documentation
> >> pages which are hard to digest. So a big +1 for splitting the Table
> API/SQL
> >> documentation up into more easily digestible pieces.
> >
> > +1
> >
> > Thanks for bringing it up
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Reorganize Table API / SQL documentation

Robert Metzger
Thank you Fabian for working on the proposal.

On Thu, May 18, 2017 at 3:51 PM, Fabian Hueske <[hidden email]> wrote:

> Thanks for starting this discussion Robert.
>
> I think with the next release the Table API / SQL should be moved up in the
> Application Development menu.
> I also though about restructuring the docs, but it won't be trivial to do
> this, IMO because there are many orthogonal aspects:
> - Stream/Batch
> - Table/SQL
> - Scala/Java
>
> and sometimes also common concepts.
> At the moment there are also many new features missing like OVER windows,
> UDAGGs, retraction, StreamTableSinks, time indicator attributes, filter
> pushdown, ...
>
> I will try to sketch a new structure in a Google Doc in the next days and
> share it in this thread.
>
> Cheers, Fabian
>
> 2017-05-18 14:03 GMT+02:00 Kostas Kloudas <[hidden email]>:
>
> > A big +1 as well.
> >
> > > On May 18, 2017, at 1:55 PM, Ufuk Celebi <[hidden email]> wrote:
> > >
> > > On Thu, May 18, 2017 at 1:52 PM, Till Rohrmann <[hidden email]>
> > wrote:
> > >> I think we have a history of creating too long monolithic
> documentation
> > >> pages which are hard to digest. So a big +1 for splitting the Table
> > API/SQL
> > >> documentation up into more easily digestible pieces.
> > >
> > > +1
> > >
> > > Thanks for bringing it up
> >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Reorganize Table API / SQL documentation

Haohui Mai
+1

The Table / SQL component has made significant progress in the last few
months (kudos to all contributors).

It is a good time to have a documentation to reflect all the changes in the
Table / SQL side.



On Thu, May 18, 2017 at 8:12 AM Robert Metzger <[hidden email]> wrote:

> Thank you Fabian for working on the proposal.
>
> On Thu, May 18, 2017 at 3:51 PM, Fabian Hueske <[hidden email]> wrote:
>
> > Thanks for starting this discussion Robert.
> >
> > I think with the next release the Table API / SQL should be moved up in
> the
> > Application Development menu.
> > I also though about restructuring the docs, but it won't be trivial to do
> > this, IMO because there are many orthogonal aspects:
> > - Stream/Batch
> > - Table/SQL
> > - Scala/Java
> >
> > and sometimes also common concepts.
> > At the moment there are also many new features missing like OVER windows,
> > UDAGGs, retraction, StreamTableSinks, time indicator attributes, filter
> > pushdown, ...
> >
> > I will try to sketch a new structure in a Google Doc in the next days and
> > share it in this thread.
> >
> > Cheers, Fabian
> >
> > 2017-05-18 14:03 GMT+02:00 Kostas Kloudas <[hidden email]>:
> >
> > > A big +1 as well.
> > >
> > > > On May 18, 2017, at 1:55 PM, Ufuk Celebi <[hidden email]> wrote:
> > > >
> > > > On Thu, May 18, 2017 at 1:52 PM, Till Rohrmann <[hidden email]
> >
> > > wrote:
> > > >> I think we have a history of creating too long monolithic
> > documentation
> > > >> pages which are hard to digest. So a big +1 for splitting the Table
> > > API/SQL
> > > >> documentation up into more easily digestible pieces.
> > > >
> > > > +1
> > > >
> > > > Thanks for bringing it up
> > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Reorganize Table API / SQL documentation

Fabian Hueske-2
Hi everybody,

I came up with a proposal for the structure of the Table API / SQL
documentation:

https://docs.google.com/document/d/1ENY8tcPadZjoZ4AQ_lRRwWiVpScDkm_4rgxIGWGT5E0/edit?usp=sharing

Feedback and comments are very welcome.
Once we agree on a structure, we can create skeletons and distribute the
work.

Cheers,
Fabian

2017-05-18 21:01 GMT+02:00 Haohui Mai <[hidden email]>:

> +1
>
> The Table / SQL component has made significant progress in the last few
> months (kudos to all contributors).
>
> It is a good time to have a documentation to reflect all the changes in the
> Table / SQL side.
>
>
>
> On Thu, May 18, 2017 at 8:12 AM Robert Metzger <[hidden email]>
> wrote:
>
> > Thank you Fabian for working on the proposal.
> >
> > On Thu, May 18, 2017 at 3:51 PM, Fabian Hueske <[hidden email]>
> wrote:
> >
> > > Thanks for starting this discussion Robert.
> > >
> > > I think with the next release the Table API / SQL should be moved up in
> > the
> > > Application Development menu.
> > > I also though about restructuring the docs, but it won't be trivial to
> do
> > > this, IMO because there are many orthogonal aspects:
> > > - Stream/Batch
> > > - Table/SQL
> > > - Scala/Java
> > >
> > > and sometimes also common concepts.
> > > At the moment there are also many new features missing like OVER
> windows,
> > > UDAGGs, retraction, StreamTableSinks, time indicator attributes, filter
> > > pushdown, ...
> > >
> > > I will try to sketch a new structure in a Google Doc in the next days
> and
> > > share it in this thread.
> > >
> > > Cheers, Fabian
> > >
> > > 2017-05-18 14:03 GMT+02:00 Kostas Kloudas <[hidden email]
> >:
> > >
> > > > A big +1 as well.
> > > >
> > > > > On May 18, 2017, at 1:55 PM, Ufuk Celebi <[hidden email]> wrote:
> > > > >
> > > > > On Thu, May 18, 2017 at 1:52 PM, Till Rohrmann <
> [hidden email]
> > >
> > > > wrote:
> > > > >> I think we have a history of creating too long monolithic
> > > documentation
> > > > >> pages which are hard to digest. So a big +1 for splitting the
> Table
> > > > API/SQL
> > > > >> documentation up into more easily digestible pieces.
> > > > >
> > > > > +1
> > > > >
> > > > > Thanks for bringing it up
> > > >
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Reorganize Table API / SQL documentation

Shaoxuan Wang
Hello Fabian,
Thanks for drafting the proposal. I like the entire organization in general
and left a few comments. I think this will be a very good kick off to
reorganize the tableAPI&SQL doc.

-shaoxuan

On Fri, May 19, 2017 at 7:06 AM, Fabian Hueske <[hidden email]> wrote:

> Hi everybody,
>
> I came up with a proposal for the structure of the Table API / SQL
> documentation:
>
> https://docs.google.com/document/d/1ENY8tcPadZjoZ4AQ_
> lRRwWiVpScDkm_4rgxIGWGT5E0/edit?usp=sharing
>
> Feedback and comments are very welcome.
> Once we agree on a structure, we can create skeletons and distribute the
> work.
>
> Cheers,
> Fabian
>
> 2017-05-18 21:01 GMT+02:00 Haohui Mai <[hidden email]>:
>
> > +1
> >
> > The Table / SQL component has made significant progress in the last few
> > months (kudos to all contributors).
> >
> > It is a good time to have a documentation to reflect all the changes in
> the
> > Table / SQL side.
> >
> >
> >
> > On Thu, May 18, 2017 at 8:12 AM Robert Metzger <[hidden email]>
> > wrote:
> >
> > > Thank you Fabian for working on the proposal.
> > >
> > > On Thu, May 18, 2017 at 3:51 PM, Fabian Hueske <[hidden email]>
> > wrote:
> > >
> > > > Thanks for starting this discussion Robert.
> > > >
> > > > I think with the next release the Table API / SQL should be moved up
> in
> > > the
> > > > Application Development menu.
> > > > I also though about restructuring the docs, but it won't be trivial
> to
> > do
> > > > this, IMO because there are many orthogonal aspects:
> > > > - Stream/Batch
> > > > - Table/SQL
> > > > - Scala/Java
> > > >
> > > > and sometimes also common concepts.
> > > > At the moment there are also many new features missing like OVER
> > windows,
> > > > UDAGGs, retraction, StreamTableSinks, time indicator attributes,
> filter
> > > > pushdown, ...
> > > >
> > > > I will try to sketch a new structure in a Google Doc in the next days
> > and
> > > > share it in this thread.
> > > >
> > > > Cheers, Fabian
> > > >
> > > > 2017-05-18 14:03 GMT+02:00 Kostas Kloudas <
> [hidden email]
> > >:
> > > >
> > > > > A big +1 as well.
> > > > >
> > > > > > On May 18, 2017, at 1:55 PM, Ufuk Celebi <[hidden email]> wrote:
> > > > > >
> > > > > > On Thu, May 18, 2017 at 1:52 PM, Till Rohrmann <
> > [hidden email]
> > > >
> > > > > wrote:
> > > > > >> I think we have a history of creating too long monolithic
> > > > documentation
> > > > > >> pages which are hard to digest. So a big +1 for splitting the
> > Table
> > > > > API/SQL
> > > > > >> documentation up into more easily digestible pieces.
> > > > > >
> > > > > > +1
> > > > > >
> > > > > > Thanks for bringing it up
> > > > >
> > > > >
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Reorganize Table API / SQL documentation

jincheng sun
Hi, Fabian,

  Thanks for the sketch. The structure is pretty well to me, And glad to
join in the discussion in google doc.

Cheers,
SunJincheng

2017-05-19 14:55 GMT+08:00 Shaoxuan Wang <[hidden email]>:

> Hello Fabian,
> Thanks for drafting the proposal. I like the entire organization in general
> and left a few comments. I think this will be a very good kick off to
> reorganize the tableAPI&SQL doc.
>
> -shaoxuan
>
> On Fri, May 19, 2017 at 7:06 AM, Fabian Hueske <[hidden email]> wrote:
>
> > Hi everybody,
> >
> > I came up with a proposal for the structure of the Table API / SQL
> > documentation:
> >
> > https://docs.google.com/document/d/1ENY8tcPadZjoZ4AQ_
> > lRRwWiVpScDkm_4rgxIGWGT5E0/edit?usp=sharing
> >
> > Feedback and comments are very welcome.
> > Once we agree on a structure, we can create skeletons and distribute the
> > work.
> >
> > Cheers,
> > Fabian
> >
> > 2017-05-18 21:01 GMT+02:00 Haohui Mai <[hidden email]>:
> >
> > > +1
> > >
> > > The Table / SQL component has made significant progress in the last few
> > > months (kudos to all contributors).
> > >
> > > It is a good time to have a documentation to reflect all the changes in
> > the
> > > Table / SQL side.
> > >
> > >
> > >
> > > On Thu, May 18, 2017 at 8:12 AM Robert Metzger <[hidden email]>
> > > wrote:
> > >
> > > > Thank you Fabian for working on the proposal.
> > > >
> > > > On Thu, May 18, 2017 at 3:51 PM, Fabian Hueske <[hidden email]>
> > > wrote:
> > > >
> > > > > Thanks for starting this discussion Robert.
> > > > >
> > > > > I think with the next release the Table API / SQL should be moved
> up
> > in
> > > > the
> > > > > Application Development menu.
> > > > > I also though about restructuring the docs, but it won't be trivial
> > to
> > > do
> > > > > this, IMO because there are many orthogonal aspects:
> > > > > - Stream/Batch
> > > > > - Table/SQL
> > > > > - Scala/Java
> > > > >
> > > > > and sometimes also common concepts.
> > > > > At the moment there are also many new features missing like OVER
> > > windows,
> > > > > UDAGGs, retraction, StreamTableSinks, time indicator attributes,
> > filter
> > > > > pushdown, ...
> > > > >
> > > > > I will try to sketch a new structure in a Google Doc in the next
> days
> > > and
> > > > > share it in this thread.
> > > > >
> > > > > Cheers, Fabian
> > > > >
> > > > > 2017-05-18 14:03 GMT+02:00 Kostas Kloudas <
> > [hidden email]
> > > >:
> > > > >
> > > > > > A big +1 as well.
> > > > > >
> > > > > > > On May 18, 2017, at 1:55 PM, Ufuk Celebi <[hidden email]>
> wrote:
> > > > > > >
> > > > > > > On Thu, May 18, 2017 at 1:52 PM, Till Rohrmann <
> > > [hidden email]
> > > > >
> > > > > > wrote:
> > > > > > >> I think we have a history of creating too long monolithic
> > > > > documentation
> > > > > > >> pages which are hard to digest. So a big +1 for splitting the
> > > Table
> > > > > > API/SQL
> > > > > > >> documentation up into more easily digestible pieces.
> > > > > > >
> > > > > > > +1
> > > > > > >
> > > > > > > Thanks for bringing it up
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Reorganize Table API / SQL documentation

Fabian Hueske-2
Hi everybody,

Thanks for the feedback. I'll go ahead and create the proposed structure
and move the content of the existing docs with comments of what needs to be
adapted.
I'll put this into branch of my Github repo and let you know when I'm done.
From there, we can distribute working on the missing parts / parts that
need adaption.

Cheers, Fabian

2017-05-19 9:44 GMT+01:00 jincheng sun <[hidden email]>:

> Hi, Fabian,
>
>   Thanks for the sketch. The structure is pretty well to me, And glad to
> join in the discussion in google doc.
>
> Cheers,
> SunJincheng
>
> 2017-05-19 14:55 GMT+08:00 Shaoxuan Wang <[hidden email]>:
>
> > Hello Fabian,
> > Thanks for drafting the proposal. I like the entire organization in
> general
> > and left a few comments. I think this will be a very good kick off to
> > reorganize the tableAPI&SQL doc.
> >
> > -shaoxuan
> >
> > On Fri, May 19, 2017 at 7:06 AM, Fabian Hueske <[hidden email]>
> wrote:
> >
> > > Hi everybody,
> > >
> > > I came up with a proposal for the structure of the Table API / SQL
> > > documentation:
> > >
> > > https://docs.google.com/document/d/1ENY8tcPadZjoZ4AQ_
> > > lRRwWiVpScDkm_4rgxIGWGT5E0/edit?usp=sharing
> > >
> > > Feedback and comments are very welcome.
> > > Once we agree on a structure, we can create skeletons and distribute
> the
> > > work.
> > >
> > > Cheers,
> > > Fabian
> > >
> > > 2017-05-18 21:01 GMT+02:00 Haohui Mai <[hidden email]>:
> > >
> > > > +1
> > > >
> > > > The Table / SQL component has made significant progress in the last
> few
> > > > months (kudos to all contributors).
> > > >
> > > > It is a good time to have a documentation to reflect all the changes
> in
> > > the
> > > > Table / SQL side.
> > > >
> > > >
> > > >
> > > > On Thu, May 18, 2017 at 8:12 AM Robert Metzger <[hidden email]>
> > > > wrote:
> > > >
> > > > > Thank you Fabian for working on the proposal.
> > > > >
> > > > > On Thu, May 18, 2017 at 3:51 PM, Fabian Hueske <[hidden email]>
> > > > wrote:
> > > > >
> > > > > > Thanks for starting this discussion Robert.
> > > > > >
> > > > > > I think with the next release the Table API / SQL should be moved
> > up
> > > in
> > > > > the
> > > > > > Application Development menu.
> > > > > > I also though about restructuring the docs, but it won't be
> trivial
> > > to
> > > > do
> > > > > > this, IMO because there are many orthogonal aspects:
> > > > > > - Stream/Batch
> > > > > > - Table/SQL
> > > > > > - Scala/Java
> > > > > >
> > > > > > and sometimes also common concepts.
> > > > > > At the moment there are also many new features missing like OVER
> > > > windows,
> > > > > > UDAGGs, retraction, StreamTableSinks, time indicator attributes,
> > > filter
> > > > > > pushdown, ...
> > > > > >
> > > > > > I will try to sketch a new structure in a Google Doc in the next
> > days
> > > > and
> > > > > > share it in this thread.
> > > > > >
> > > > > > Cheers, Fabian
> > > > > >
> > > > > > 2017-05-18 14:03 GMT+02:00 Kostas Kloudas <
> > > [hidden email]
> > > > >:
> > > > > >
> > > > > > > A big +1 as well.
> > > > > > >
> > > > > > > > On May 18, 2017, at 1:55 PM, Ufuk Celebi <[hidden email]>
> > wrote:
> > > > > > > >
> > > > > > > > On Thu, May 18, 2017 at 1:52 PM, Till Rohrmann <
> > > > [hidden email]
> > > > > >
> > > > > > > wrote:
> > > > > > > >> I think we have a history of creating too long monolithic
> > > > > > documentation
> > > > > > > >> pages which are hard to digest. So a big +1 for splitting
> the
> > > > Table
> > > > > > > API/SQL
> > > > > > > >> documentation up into more easily digestible pieces.
> > > > > > > >
> > > > > > > > +1
> > > > > > > >
> > > > > > > > Thanks for bringing it up
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Reorganize Table API / SQL documentation

Fabian Hueske-2
Hi everybody,

I prepared a branch that creates the proposed structure and copied the
existing documentation into the corresponding pages / sections.
There are plenty of gaps that need to be filled or reworked.

--> https://github.com/fhueske/flink/tree/tableDocs

How do we go on from here?
I think the easiest would be if everybody who's interested in working on
the documentation picks a page and prepares a PR against my branch (we
could also push this into a feature branch in the Flink repository if
somebody prefers that). The PRs are cross-checked and we merge everything
into to the master when the docs are ready.

Any opinions or other proposals?

Cheers, Fabian

2017-05-23 10:31 GMT+01:00 Fabian Hueske <[hidden email]>:

> Hi everybody,
>
> Thanks for the feedback. I'll go ahead and create the proposed structure
> and move the content of the existing docs with comments of what needs to be
> adapted.
> I'll put this into branch of my Github repo and let you know when I'm done.
> From there, we can distribute working on the missing parts / parts that
> need adaption.
>
> Cheers, Fabian
>
> 2017-05-19 9:44 GMT+01:00 jincheng sun <[hidden email]>:
>
>> Hi, Fabian,
>>
>>   Thanks for the sketch. The structure is pretty well to me, And glad to
>> join in the discussion in google doc.
>>
>> Cheers,
>> SunJincheng
>>
>> 2017-05-19 14:55 GMT+08:00 Shaoxuan Wang <[hidden email]>:
>>
>> > Hello Fabian,
>> > Thanks for drafting the proposal. I like the entire organization in
>> general
>> > and left a few comments. I think this will be a very good kick off to
>> > reorganize the tableAPI&SQL doc.
>> >
>> > -shaoxuan
>> >
>> > On Fri, May 19, 2017 at 7:06 AM, Fabian Hueske <[hidden email]>
>> wrote:
>> >
>> > > Hi everybody,
>> > >
>> > > I came up with a proposal for the structure of the Table API / SQL
>> > > documentation:
>> > >
>> > > https://docs.google.com/document/d/1ENY8tcPadZjoZ4AQ_
>> > > lRRwWiVpScDkm_4rgxIGWGT5E0/edit?usp=sharing
>> > >
>> > > Feedback and comments are very welcome.
>> > > Once we agree on a structure, we can create skeletons and distribute
>> the
>> > > work.
>> > >
>> > > Cheers,
>> > > Fabian
>> > >
>> > > 2017-05-18 21:01 GMT+02:00 Haohui Mai <[hidden email]>:
>> > >
>> > > > +1
>> > > >
>> > > > The Table / SQL component has made significant progress in the last
>> few
>> > > > months (kudos to all contributors).
>> > > >
>> > > > It is a good time to have a documentation to reflect all the
>> changes in
>> > > the
>> > > > Table / SQL side.
>> > > >
>> > > >
>> > > >
>> > > > On Thu, May 18, 2017 at 8:12 AM Robert Metzger <[hidden email]
>> >
>> > > > wrote:
>> > > >
>> > > > > Thank you Fabian for working on the proposal.
>> > > > >
>> > > > > On Thu, May 18, 2017 at 3:51 PM, Fabian Hueske <[hidden email]
>> >
>> > > > wrote:
>> > > > >
>> > > > > > Thanks for starting this discussion Robert.
>> > > > > >
>> > > > > > I think with the next release the Table API / SQL should be
>> moved
>> > up
>> > > in
>> > > > > the
>> > > > > > Application Development menu.
>> > > > > > I also though about restructuring the docs, but it won't be
>> trivial
>> > > to
>> > > > do
>> > > > > > this, IMO because there are many orthogonal aspects:
>> > > > > > - Stream/Batch
>> > > > > > - Table/SQL
>> > > > > > - Scala/Java
>> > > > > >
>> > > > > > and sometimes also common concepts.
>> > > > > > At the moment there are also many new features missing like OVER
>> > > > windows,
>> > > > > > UDAGGs, retraction, StreamTableSinks, time indicator attributes,
>> > > filter
>> > > > > > pushdown, ...
>> > > > > >
>> > > > > > I will try to sketch a new structure in a Google Doc in the next
>> > days
>> > > > and
>> > > > > > share it in this thread.
>> > > > > >
>> > > > > > Cheers, Fabian
>> > > > > >
>> > > > > > 2017-05-18 14:03 GMT+02:00 Kostas Kloudas <
>> > > [hidden email]
>> > > > >:
>> > > > > >
>> > > > > > > A big +1 as well.
>> > > > > > >
>> > > > > > > > On May 18, 2017, at 1:55 PM, Ufuk Celebi <[hidden email]>
>> > wrote:
>> > > > > > > >
>> > > > > > > > On Thu, May 18, 2017 at 1:52 PM, Till Rohrmann <
>> > > > [hidden email]
>> > > > > >
>> > > > > > > wrote:
>> > > > > > > >> I think we have a history of creating too long monolithic
>> > > > > > documentation
>> > > > > > > >> pages which are hard to digest. So a big +1 for splitting
>> the
>> > > > Table
>> > > > > > > API/SQL
>> > > > > > > >> documentation up into more easily digestible pieces.
>> > > > > > > >
>> > > > > > > > +1
>> > > > > > > >
>> > > > > > > > Thanks for bringing it up
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Reorganize Table API / SQL documentation

Fabian Hueske-2
Hi everybody,

I pushed the branch to the ASF Flink repository as a feature branch to keep
all PRs in one place:

-->  https://github.com/apache/flink/tree/tableDocs

Thanks,
Fabian

2017-05-23 16:25 GMT+01:00 Fabian Hueske <[hidden email]>:

> Hi everybody,
>
> I prepared a branch that creates the proposed structure and copied the
> existing documentation into the corresponding pages / sections.
> There are plenty of gaps that need to be filled or reworked.
>
> --> https://github.com/fhueske/flink/tree/tableDocs
>
> How do we go on from here?
> I think the easiest would be if everybody who's interested in working on
> the documentation picks a page and prepares a PR against my branch (we
> could also push this into a feature branch in the Flink repository if
> somebody prefers that). The PRs are cross-checked and we merge everything
> into to the master when the docs are ready.
>
> Any opinions or other proposals?
>
> Cheers, Fabian
>
> 2017-05-23 10:31 GMT+01:00 Fabian Hueske <[hidden email]>:
>
>> Hi everybody,
>>
>> Thanks for the feedback. I'll go ahead and create the proposed structure
>> and move the content of the existing docs with comments of what needs to be
>> adapted.
>> I'll put this into branch of my Github repo and let you know when I'm
>> done.
>> From there, we can distribute working on the missing parts / parts that
>> need adaption.
>>
>> Cheers, Fabian
>>
>> 2017-05-19 9:44 GMT+01:00 jincheng sun <[hidden email]>:
>>
>>> Hi, Fabian,
>>>
>>>   Thanks for the sketch. The structure is pretty well to me, And glad to
>>> join in the discussion in google doc.
>>>
>>> Cheers,
>>> SunJincheng
>>>
>>> 2017-05-19 14:55 GMT+08:00 Shaoxuan Wang <[hidden email]>:
>>>
>>> > Hello Fabian,
>>> > Thanks for drafting the proposal. I like the entire organization in
>>> general
>>> > and left a few comments. I think this will be a very good kick off to
>>> > reorganize the tableAPI&SQL doc.
>>> >
>>> > -shaoxuan
>>> >
>>> > On Fri, May 19, 2017 at 7:06 AM, Fabian Hueske <[hidden email]>
>>> wrote:
>>> >
>>> > > Hi everybody,
>>> > >
>>> > > I came up with a proposal for the structure of the Table API / SQL
>>> > > documentation:
>>> > >
>>> > > https://docs.google.com/document/d/1ENY8tcPadZjoZ4AQ_
>>> > > lRRwWiVpScDkm_4rgxIGWGT5E0/edit?usp=sharing
>>> > >
>>> > > Feedback and comments are very welcome.
>>> > > Once we agree on a structure, we can create skeletons and distribute
>>> the
>>> > > work.
>>> > >
>>> > > Cheers,
>>> > > Fabian
>>> > >
>>> > > 2017-05-18 21:01 GMT+02:00 Haohui Mai <[hidden email]>:
>>> > >
>>> > > > +1
>>> > > >
>>> > > > The Table / SQL component has made significant progress in the
>>> last few
>>> > > > months (kudos to all contributors).
>>> > > >
>>> > > > It is a good time to have a documentation to reflect all the
>>> changes in
>>> > > the
>>> > > > Table / SQL side.
>>> > > >
>>> > > >
>>> > > >
>>> > > > On Thu, May 18, 2017 at 8:12 AM Robert Metzger <
>>> [hidden email]>
>>> > > > wrote:
>>> > > >
>>> > > > > Thank you Fabian for working on the proposal.
>>> > > > >
>>> > > > > On Thu, May 18, 2017 at 3:51 PM, Fabian Hueske <
>>> [hidden email]>
>>> > > > wrote:
>>> > > > >
>>> > > > > > Thanks for starting this discussion Robert.
>>> > > > > >
>>> > > > > > I think with the next release the Table API / SQL should be
>>> moved
>>> > up
>>> > > in
>>> > > > > the
>>> > > > > > Application Development menu.
>>> > > > > > I also though about restructuring the docs, but it won't be
>>> trivial
>>> > > to
>>> > > > do
>>> > > > > > this, IMO because there are many orthogonal aspects:
>>> > > > > > - Stream/Batch
>>> > > > > > - Table/SQL
>>> > > > > > - Scala/Java
>>> > > > > >
>>> > > > > > and sometimes also common concepts.
>>> > > > > > At the moment there are also many new features missing like
>>> OVER
>>> > > > windows,
>>> > > > > > UDAGGs, retraction, StreamTableSinks, time indicator
>>> attributes,
>>> > > filter
>>> > > > > > pushdown, ...
>>> > > > > >
>>> > > > > > I will try to sketch a new structure in a Google Doc in the
>>> next
>>> > days
>>> > > > and
>>> > > > > > share it in this thread.
>>> > > > > >
>>> > > > > > Cheers, Fabian
>>> > > > > >
>>> > > > > > 2017-05-18 14:03 GMT+02:00 Kostas Kloudas <
>>> > > [hidden email]
>>> > > > >:
>>> > > > > >
>>> > > > > > > A big +1 as well.
>>> > > > > > >
>>> > > > > > > > On May 18, 2017, at 1:55 PM, Ufuk Celebi <[hidden email]>
>>> > wrote:
>>> > > > > > > >
>>> > > > > > > > On Thu, May 18, 2017 at 1:52 PM, Till Rohrmann <
>>> > > > [hidden email]
>>> > > > > >
>>> > > > > > > wrote:
>>> > > > > > > >> I think we have a history of creating too long monolithic
>>> > > > > > documentation
>>> > > > > > > >> pages which are hard to digest. So a big +1 for splitting
>>> the
>>> > > > Table
>>> > > > > > > API/SQL
>>> > > > > > > >> documentation up into more easily digestible pieces.
>>> > > > > > > >
>>> > > > > > > > +1
>>> > > > > > > >
>>> > > > > > > > Thanks for bringing it up
>>> > > > > > >
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>>
>>
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Reorganize Table API / SQL documentation

Fabian Hueske-2
Hi everybody,

I added a couple of subissues to
https://issues.apache.org/jira/browse/FLINK-5354 and started to work on the
first two.
I'll open PRs for those soon.

Please pick an issue and assign it to yourself if you want to help with the
Table API / SQL documentation.

Thanks,
Fabian


2017-05-23 17:49 GMT+02:00 Fabian Hueske <[hidden email]>:

> Hi everybody,
>
> I pushed the branch to the ASF Flink repository as a feature branch to
> keep all PRs in one place:
>
> -->  https://github.com/apache/flink/tree/tableDocs
>
> Thanks,
> Fabian
>
> 2017-05-23 16:25 GMT+01:00 Fabian Hueske <[hidden email]>:
>
>> Hi everybody,
>>
>> I prepared a branch that creates the proposed structure and copied the
>> existing documentation into the corresponding pages / sections.
>> There are plenty of gaps that need to be filled or reworked.
>>
>> --> https://github.com/fhueske/flink/tree/tableDocs
>>
>> How do we go on from here?
>> I think the easiest would be if everybody who's interested in working on
>> the documentation picks a page and prepares a PR against my branch (we
>> could also push this into a feature branch in the Flink repository if
>> somebody prefers that). The PRs are cross-checked and we merge everything
>> into to the master when the docs are ready.
>>
>> Any opinions or other proposals?
>>
>> Cheers, Fabian
>>
>> 2017-05-23 10:31 GMT+01:00 Fabian Hueske <[hidden email]>:
>>
>>> Hi everybody,
>>>
>>> Thanks for the feedback. I'll go ahead and create the proposed structure
>>> and move the content of the existing docs with comments of what needs to be
>>> adapted.
>>> I'll put this into branch of my Github repo and let you know when I'm
>>> done.
>>> From there, we can distribute working on the missing parts / parts that
>>> need adaption.
>>>
>>> Cheers, Fabian
>>>
>>> 2017-05-19 9:44 GMT+01:00 jincheng sun <[hidden email]>:
>>>
>>>> Hi, Fabian,
>>>>
>>>>   Thanks for the sketch. The structure is pretty well to me, And glad to
>>>> join in the discussion in google doc.
>>>>
>>>> Cheers,
>>>> SunJincheng
>>>>
>>>> 2017-05-19 14:55 GMT+08:00 Shaoxuan Wang <[hidden email]>:
>>>>
>>>> > Hello Fabian,
>>>> > Thanks for drafting the proposal. I like the entire organization in
>>>> general
>>>> > and left a few comments. I think this will be a very good kick off to
>>>> > reorganize the tableAPI&SQL doc.
>>>> >
>>>> > -shaoxuan
>>>> >
>>>> > On Fri, May 19, 2017 at 7:06 AM, Fabian Hueske <[hidden email]>
>>>> wrote:
>>>> >
>>>> > > Hi everybody,
>>>> > >
>>>> > > I came up with a proposal for the structure of the Table API / SQL
>>>> > > documentation:
>>>> > >
>>>> > > https://docs.google.com/document/d/1ENY8tcPadZjoZ4AQ_
>>>> > > lRRwWiVpScDkm_4rgxIGWGT5E0/edit?usp=sharing
>>>> > >
>>>> > > Feedback and comments are very welcome.
>>>> > > Once we agree on a structure, we can create skeletons and
>>>> distribute the
>>>> > > work.
>>>> > >
>>>> > > Cheers,
>>>> > > Fabian
>>>> > >
>>>> > > 2017-05-18 21:01 GMT+02:00 Haohui Mai <[hidden email]>:
>>>> > >
>>>> > > > +1
>>>> > > >
>>>> > > > The Table / SQL component has made significant progress in the
>>>> last few
>>>> > > > months (kudos to all contributors).
>>>> > > >
>>>> > > > It is a good time to have a documentation to reflect all the
>>>> changes in
>>>> > > the
>>>> > > > Table / SQL side.
>>>> > > >
>>>> > > >
>>>> > > >
>>>> > > > On Thu, May 18, 2017 at 8:12 AM Robert Metzger <
>>>> [hidden email]>
>>>> > > > wrote:
>>>> > > >
>>>> > > > > Thank you Fabian for working on the proposal.
>>>> > > > >
>>>> > > > > On Thu, May 18, 2017 at 3:51 PM, Fabian Hueske <
>>>> [hidden email]>
>>>> > > > wrote:
>>>> > > > >
>>>> > > > > > Thanks for starting this discussion Robert.
>>>> > > > > >
>>>> > > > > > I think with the next release the Table API / SQL should be
>>>> moved
>>>> > up
>>>> > > in
>>>> > > > > the
>>>> > > > > > Application Development menu.
>>>> > > > > > I also though about restructuring the docs, but it won't be
>>>> trivial
>>>> > > to
>>>> > > > do
>>>> > > > > > this, IMO because there are many orthogonal aspects:
>>>> > > > > > - Stream/Batch
>>>> > > > > > - Table/SQL
>>>> > > > > > - Scala/Java
>>>> > > > > >
>>>> > > > > > and sometimes also common concepts.
>>>> > > > > > At the moment there are also many new features missing like
>>>> OVER
>>>> > > > windows,
>>>> > > > > > UDAGGs, retraction, StreamTableSinks, time indicator
>>>> attributes,
>>>> > > filter
>>>> > > > > > pushdown, ...
>>>> > > > > >
>>>> > > > > > I will try to sketch a new structure in a Google Doc in the
>>>> next
>>>> > days
>>>> > > > and
>>>> > > > > > share it in this thread.
>>>> > > > > >
>>>> > > > > > Cheers, Fabian
>>>> > > > > >
>>>> > > > > > 2017-05-18 14:03 GMT+02:00 Kostas Kloudas <
>>>> > > [hidden email]
>>>> > > > >:
>>>> > > > > >
>>>> > > > > > > A big +1 as well.
>>>> > > > > > >
>>>> > > > > > > > On May 18, 2017, at 1:55 PM, Ufuk Celebi <[hidden email]>
>>>> > wrote:
>>>> > > > > > > >
>>>> > > > > > > > On Thu, May 18, 2017 at 1:52 PM, Till Rohrmann <
>>>> > > > [hidden email]
>>>> > > > > >
>>>> > > > > > > wrote:
>>>> > > > > > > >> I think we have a history of creating too long monolithic
>>>> > > > > > documentation
>>>> > > > > > > >> pages which are hard to digest. So a big +1 for
>>>> splitting the
>>>> > > > Table
>>>> > > > > > > API/SQL
>>>> > > > > > > >> documentation up into more easily digestible pieces.
>>>> > > > > > > >
>>>> > > > > > > > +1
>>>> > > > > > > >
>>>> > > > > > > > Thanks for bringing it up
>>>> > > > > > >
>>>> > > > > > >
>>>> > > > > >
>>>> > > > >
>>>> > > >
>>>> > >
>>>> >
>>>>
>>>
>>>
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Reorganize Table API / SQL documentation

Fabian Hueske-2
Hi,

I merged the feature branch back to the master and the release-1.3 branch
and deleted it.
There are still a few gaps in the docs that need to be filled (there are
JIRAs for those), but the overall structure looks good IMO and most aspects
of the APIs are covered.

Cheers, Fabian

2017-05-28 15:21 GMT+02:00 Fabian Hueske <[hidden email]>:

> Hi everybody,
>
> I added a couple of subissues to https://issues.apache.org/
> jira/browse/FLINK-5354 and started to work on the first two.
> I'll open PRs for those soon.
>
> Please pick an issue and assign it to yourself if you want to help with
> the Table API / SQL documentation.
>
> Thanks,
> Fabian
>
>
> 2017-05-23 17:49 GMT+02:00 Fabian Hueske <[hidden email]>:
>
>> Hi everybody,
>>
>> I pushed the branch to the ASF Flink repository as a feature branch to
>> keep all PRs in one place:
>>
>> -->  https://github.com/apache/flink/tree/tableDocs
>>
>> Thanks,
>> Fabian
>>
>> 2017-05-23 16:25 GMT+01:00 Fabian Hueske <[hidden email]>:
>>
>>> Hi everybody,
>>>
>>> I prepared a branch that creates the proposed structure and copied the
>>> existing documentation into the corresponding pages / sections.
>>> There are plenty of gaps that need to be filled or reworked.
>>>
>>> --> https://github.com/fhueske/flink/tree/tableDocs
>>>
>>> How do we go on from here?
>>> I think the easiest would be if everybody who's interested in working on
>>> the documentation picks a page and prepares a PR against my branch (we
>>> could also push this into a feature branch in the Flink repository if
>>> somebody prefers that). The PRs are cross-checked and we merge everything
>>> into to the master when the docs are ready.
>>>
>>> Any opinions or other proposals?
>>>
>>> Cheers, Fabian
>>>
>>> 2017-05-23 10:31 GMT+01:00 Fabian Hueske <[hidden email]>:
>>>
>>>> Hi everybody,
>>>>
>>>> Thanks for the feedback. I'll go ahead and create the proposed
>>>> structure and move the content of the existing docs with comments of what
>>>> needs to be adapted.
>>>> I'll put this into branch of my Github repo and let you know when I'm
>>>> done.
>>>> From there, we can distribute working on the missing parts / parts that
>>>> need adaption.
>>>>
>>>> Cheers, Fabian
>>>>
>>>> 2017-05-19 9:44 GMT+01:00 jincheng sun <[hidden email]>:
>>>>
>>>>> Hi, Fabian,
>>>>>
>>>>>   Thanks for the sketch. The structure is pretty well to me, And glad
>>>>> to
>>>>> join in the discussion in google doc.
>>>>>
>>>>> Cheers,
>>>>> SunJincheng
>>>>>
>>>>> 2017-05-19 14:55 GMT+08:00 Shaoxuan Wang <[hidden email]>:
>>>>>
>>>>> > Hello Fabian,
>>>>> > Thanks for drafting the proposal. I like the entire organization in
>>>>> general
>>>>> > and left a few comments. I think this will be a very good kick off to
>>>>> > reorganize the tableAPI&SQL doc.
>>>>> >
>>>>> > -shaoxuan
>>>>> >
>>>>> > On Fri, May 19, 2017 at 7:06 AM, Fabian Hueske <[hidden email]>
>>>>> wrote:
>>>>> >
>>>>> > > Hi everybody,
>>>>> > >
>>>>> > > I came up with a proposal for the structure of the Table API / SQL
>>>>> > > documentation:
>>>>> > >
>>>>> > > https://docs.google.com/document/d/1ENY8tcPadZjoZ4AQ_
>>>>> > > lRRwWiVpScDkm_4rgxIGWGT5E0/edit?usp=sharing
>>>>> > >
>>>>> > > Feedback and comments are very welcome.
>>>>> > > Once we agree on a structure, we can create skeletons and
>>>>> distribute the
>>>>> > > work.
>>>>> > >
>>>>> > > Cheers,
>>>>> > > Fabian
>>>>> > >
>>>>> > > 2017-05-18 21:01 GMT+02:00 Haohui Mai <[hidden email]>:
>>>>> > >
>>>>> > > > +1
>>>>> > > >
>>>>> > > > The Table / SQL component has made significant progress in the
>>>>> last few
>>>>> > > > months (kudos to all contributors).
>>>>> > > >
>>>>> > > > It is a good time to have a documentation to reflect all the
>>>>> changes in
>>>>> > > the
>>>>> > > > Table / SQL side.
>>>>> > > >
>>>>> > > >
>>>>> > > >
>>>>> > > > On Thu, May 18, 2017 at 8:12 AM Robert Metzger <
>>>>> [hidden email]>
>>>>> > > > wrote:
>>>>> > > >
>>>>> > > > > Thank you Fabian for working on the proposal.
>>>>> > > > >
>>>>> > > > > On Thu, May 18, 2017 at 3:51 PM, Fabian Hueske <
>>>>> [hidden email]>
>>>>> > > > wrote:
>>>>> > > > >
>>>>> > > > > > Thanks for starting this discussion Robert.
>>>>> > > > > >
>>>>> > > > > > I think with the next release the Table API / SQL should be
>>>>> moved
>>>>> > up
>>>>> > > in
>>>>> > > > > the
>>>>> > > > > > Application Development menu.
>>>>> > > > > > I also though about restructuring the docs, but it won't be
>>>>> trivial
>>>>> > > to
>>>>> > > > do
>>>>> > > > > > this, IMO because there are many orthogonal aspects:
>>>>> > > > > > - Stream/Batch
>>>>> > > > > > - Table/SQL
>>>>> > > > > > - Scala/Java
>>>>> > > > > >
>>>>> > > > > > and sometimes also common concepts.
>>>>> > > > > > At the moment there are also many new features missing like
>>>>> OVER
>>>>> > > > windows,
>>>>> > > > > > UDAGGs, retraction, StreamTableSinks, time indicator
>>>>> attributes,
>>>>> > > filter
>>>>> > > > > > pushdown, ...
>>>>> > > > > >
>>>>> > > > > > I will try to sketch a new structure in a Google Doc in the
>>>>> next
>>>>> > days
>>>>> > > > and
>>>>> > > > > > share it in this thread.
>>>>> > > > > >
>>>>> > > > > > Cheers, Fabian
>>>>> > > > > >
>>>>> > > > > > 2017-05-18 14:03 GMT+02:00 Kostas Kloudas <
>>>>> > > [hidden email]
>>>>> > > > >:
>>>>> > > > > >
>>>>> > > > > > > A big +1 as well.
>>>>> > > > > > >
>>>>> > > > > > > > On May 18, 2017, at 1:55 PM, Ufuk Celebi <[hidden email]
>>>>> >
>>>>> > wrote:
>>>>> > > > > > > >
>>>>> > > > > > > > On Thu, May 18, 2017 at 1:52 PM, Till Rohrmann <
>>>>> > > > [hidden email]
>>>>> > > > > >
>>>>> > > > > > > wrote:
>>>>> > > > > > > >> I think we have a history of creating too long
>>>>> monolithic
>>>>> > > > > > documentation
>>>>> > > > > > > >> pages which are hard to digest. So a big +1 for
>>>>> splitting the
>>>>> > > > Table
>>>>> > > > > > > API/SQL
>>>>> > > > > > > >> documentation up into more easily digestible pieces.
>>>>> > > > > > > >
>>>>> > > > > > > > +1
>>>>> > > > > > > >
>>>>> > > > > > > > Thanks for bringing it up
>>>>> > > > > > >
>>>>> > > > > > >
>>>>> > > > > >
>>>>> > > > >
>>>>> > > >
>>>>> > >
>>>>> >
>>>>>
>>>>
>>>>
>>>
>>
>