[Tf-aai] Filtering suspicious IdPs in CLARIN SPF

[Jozef Misutka]

And we have a case study from couple of hours ago,
https://trac.clarin.eu/ticket/1008#ticket

Steps 1. and 2. are done,
for 3. we do not have the requirements yet but anyway, please comment in
the trac ticket.

2Andre: (I am not trac admin) but I suggest you
a) create specific component in trac (AAI IdP Review?) and update the
tickets;
b) create the parallel trac page (I am not sure I know what you want there)
and include listing of all bugs that are open with specific component on
that page.

Best,
Jozef


On 10 May 2017 at 17:53, André Moreira <andre at clarin.eu> wrote:

> Hi Jozef,
>
> > And yes, I was talking only about the process flow for reviewing an IdP
> and possibly blacklisting it.
>
> Clear.
>
> > Because you are? responsible for clarin's pyff we have to work on this
> together or rather you have to agree to it :).
>
> Fair enough!
>
> > 1. someone (here, the term someone is ok) finds suspicious IdP
> > 2. creates ticket in TRAC
> > 3. someone (here, the term someone should be better defined - tf-aai?)
> reviews the requirements and comments whether any violations have been
> found.
> > 4. Andre closes the ticket, if there is a violation he will do it after
> clarin's pyff is updated.
>
>
> I still need the advantages of keeping a parallel trac page with the
> overall status of the blacklisting activities, especially since there is
> the potential for multiple people to handle step 3, this page can come
> handy. It is also a nice place to write the workflow we are discussing here
> in a wiki style.
> Thus I would modify 3 and 4:
> 3. someone (here, the term someone should be better defined - tf-aai?)
> reviews the requirements and comments whether any violations have been
> found updating the IdP blacklist status page on trac accordingly.
> 4. André closes the ticket and updates the blacklist status page
> accordingly, if there is a violation he will do it after clarin's pyff is
> updated.
>
> > One comment though, I do not think the xpath selection in this format is
> sustainable in the long run. But it is up to you Andre to decide what to do
> about it if anything.
>
> This has also been one of my remarks when I was introduced to the system,
> so I fully agree!
> My current idea is to create a small script that will convert a text file
> with all blacklisted IdPs (one per line) into the necessary XPaths and
> insert them into the job_c.fd code. I would like to hear if someone as an
> alternative idea.
>
> If more people are interested we can arrange a tf-aai meeting during the
> CLARIN centre meeting on Wednesday the 17th after lunch, since the
> afternoon is supposed to be dedicated to taskforce meetings anyway. Also
> feel free to grab me for an informal chat about this at anytime during the
> centre meeting.
>
>
> Regards,
> ----
> André Moreira
> CLARIN ERIC
> https://www.clarin.eu
>
>
>
> > On 2 May 2017, at 19:42, Jozef Misutka <misutka at ufal.mff.cuni.cz> wrote:
> >
> > Andre, you answered everything, thank you.
> >
> > And yes, I was talking only about the process flow for reviewing an IdP
> and possibly blacklisting it.
> > Because you are? responsible for clarin's pyff we have to work on this
> together or rather you have to agree to it :).
> >
> > The VC or possibly the meeting at the dev conf. can help us defining the
> baseline requirements. Besides that, we have this:
> > 1. someone (here, the term someone is ok) finds suspicious IdP
> > 2. creates ticket in TRAC
> > 3. someone (here, the term someone should be better defined - tf-aai?)
> reviews the requirements and comments whether any violations have been
> found.
> > 4. Andre closes the ticket, if there is a violation he will do it after
> clarin's pyff is updated.
> >
> > One comment though, I do not think the xpath selection in this format is
> sustainable in the long run. But it is up to you Andre to decide what to do
> about it if anything.
> >
> > Best,
> > Jozef
> >
> >
> >
> >
> > On 1 May 2017 at 17:20, André Moreira <andre at clarin.eu> wrote:
> > Hi Jozef,
> >
> >  My apologies for the late answer.
> >  I must admit I am a bit out of context here; is your idea to streamline
> a workflow for the detection and blacklisting of suspicious IdPs, or just
> the later?
> >
> > >> 2. In 1a) or 1b) someone finds a suspicious IdP and CLARIN SPF should
> be notified about it
> > >>    QUESTION: what is the best/preferred way to notify CLARIN SPF -
> trac? email?
> >
> >
> > My preference is via a trac ticket, so we could have both: an immediate
> notification sent by email, plus the status and history of the actions
> taken.
> > Furthermore, I think it is also important to keep an inventory of the
> all the blacklisted IdPs and the respective reasoning for their
> blacklisting. So I suggest we create a trac page for the 'IdPs blacklist
> status’, but that the initial reports of misbehaving IdPs are done with the
> creation of trac tickets. Then the operator handling one of these tickets,
> would be responsible for updating the 'IdPs blacklist status’ page.
> >
> >
> > >> 4. SPF will add this IdP into the filters
> > >>   QUESTION: is filtering (blacklisting) of IdPs in CLARIN SPF
> happening before they are published to https://infra.clarin.eu/aai/
> prod_md_about_spf_idps.xml and if so then where?
> > >>   ACTION POINT: make filtered IdPs public (will be decided what is
> the best way [1])
> >
> > Yes, filtering happens before publishing. You can see the PyFF code we
> are using here: https://github.com/clarin-eric/pyFF_config/blob/master/
> job_c.fd (see ’select’ XPath expressions).
> >
> >
> > But again, I feel like I am not fully contextualised here, so I might
> not be giving you the answers you were looking for. In that case, if it
> suits you, we could schedule a skype call so we can have a more agile
> discussion?
> >
> >
> > Anyhow I hope this helps,
> > Kind regards,
> > ----
> > André Moreira
> > CLARIN ERIC
> > https://www.clarin.eu
> >
> >
> >
> > > On 18 Apr 2017, at 12:29, Jozef Misutka <misutka at ufal.mff.cuni.cz>
> wrote:
> > >
> > > Hi Andre,
> > >
> > > (cc-ing tf-aai@ too for comments)
> > >
> > > we should have a more formal and public way how to blacklist IdPs from
> CLARIN SPF. We have the tools, so we really need only minor changes and
> better documentation. In reality, this will be very rare but should be
> documented nevertheless.
> > >
> > > The flow is like this:
> > >
> > > 1. User authenticates to one of CLARIN SPF SPs
> > >    a) we have information about the IdP in https://lindat.mff.cuni.cz/
> services/aaggreg/ so we can check if it looks suspicious or not;
> > >    b) different web applications inform about new users (e.g.,
> clarin-dspace sends an email) so we can even check if the attributes seem
> ok.
> > >    ACTION POINT (optional): email new IdPs from attribute aggregator
> > >
> > > 2. In 1a) or 1b) someone finds a suspicious IdP and CLARIN SPF should
> be notified about it
> > >    QUESTION: what is the best/preferred way to notify CLARIN SPF -
> trac? email?
> > >
> > > 3. CLARIN evaluates if that IdP is really problematic or not
> > >    ACTION POINT: define a problematic/non problematic IdP e.g.,
> problematic IdP is the one that does not validate its users more that by
> sending emails
> > >    ACTION POINT: make this definition public (will be decided what is
> the best way [1])
> > >
> > > 4. SPF will add this IdP into the filters
> > >   QUESTION: is filtering (blacklisting) of IdPs in CLARIN SPF
> happening before they are published to https://infra.clarin.eu/aai/
> prod_md_about_spf_idps.xml and if so then where?
> > >   ACTION POINT: make filtered IdPs public (will be decided what is the
> best way [1])
> > >
> > >
> > > To all, please feel free to comment, share practices etc. At LINDAT,
> we have the following page
> > > https://lindat.mff.cuni.cz/en/how-do-i-sign-up (see section Supported
> Organisations)
> > >
> > > Best,
> > > Jozef
> > >
> > >
> > > [1] In my todo list is still to prepare a short CLARIN AAI
> requirements document where we could add it and publish that document.
> >
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clarin.eu/cgi-bin/mailman/private/tf-aai/attachments/20170512/db4ecb4a/attachment.htm>