[Tf-aai] Filtering suspicious IdPs in CLARIN SPF

Thu May 25 09:57:22 CEST 2017

On 12 May 2017 at 10:24, Jozef Misutka <misutka at ufal.mff.cuni.cz> wrote:

> And we have a case study from couple of hours ago,
> https://trac.clarin.eu/ticket/1008#ticket
>

Anyone?
Seems trac is not working as fast as (I would have) expected.

Jozef

>
>
> Steps 1. and 2. are done,
> for 3. we do not have the requirements yet but anyway, please comment in
> the trac ticket.
>
> 2Andre: (I am not trac admin) but I suggest you
> a) create specific component in trac (AAI IdP Review?) and update the
> tickets;
> b) create the parallel trac page (I am not sure I know what you want
> there) and include listing of all bugs that are open with specific
> component on that page.
>
> Best,
> Jozef
>
>
> On 10 May 2017 at 17:53, André Moreira <andre at clarin.eu> wrote:
>
>> Hi Jozef,
>>
>> > And yes, I was talking only about the process flow for reviewing an IdP
>> and possibly blacklisting it.
>>
>> Clear.
>>
>> > Because you are? responsible for clarin's pyff we have to work on this
>> together or rather you have to agree to it :).
>>
>> Fair enough!
>>
>> > 1. someone (here, the term someone is ok) finds suspicious IdP
>> > 2. creates ticket in TRAC
>> > 3. someone (here, the term someone should be better defined - tf-aai?)
>> reviews the requirements and comments whether any violations have been
>> found.
>> > 4. Andre closes the ticket, if there is a violation he will do it after
>> clarin's pyff is updated.
>>
>>
>> I still need the advantages of keeping a parallel trac page with the
>> overall status of the blacklisting activities, especially since there is
>> the potential for multiple people to handle step 3, this page can come
>> handy. It is also a nice place to write the workflow we are discussing here
>> in a wiki style.
>> Thus I would modify 3 and 4:
>> 3. someone (here, the term someone should be better defined - tf-aai?)
>> reviews the requirements and comments whether any violations have been
>> found updating the IdP blacklist status page on trac accordingly.
>> 4. André closes the ticket and updates the blacklist status page
>> accordingly, if there is a violation he will do it after clarin's pyff is
>> updated.
>>
>> > One comment though, I do not think the xpath selection in this format
>> is sustainable in the long run. But it is up to you Andre to decide what to
>> do about it if anything.
>>
>> This has also been one of my remarks when I was introduced to the system,
>> so I fully agree!
>> My current idea is to create a small script that will convert a text file
>> with all blacklisted IdPs (one per line) into the necessary XPaths and
>> insert them into the job_c.fd code. I would like to hear if someone as an
>> alternative idea.
>>
>> If more people are interested we can arrange a tf-aai meeting during the
>> CLARIN centre meeting on Wednesday the 17th after lunch, since the
>> afternoon is supposed to be dedicated to taskforce meetings anyway. Also
>> feel free to grab me for an informal chat about this at anytime during the
>> centre meeting.
>>
>>
>> Regards,
>> ----
>> André Moreira
>> CLARIN ERIC
>> https://www.clarin.eu
>>
>>
>>
>> > On 2 May 2017, at 19:42, Jozef Misutka <misutka at ufal.mff.cuni.cz>
>> wrote:
>> >
>> > Andre, you answered everything, thank you.
>> >
>> > And yes, I was talking only about the process flow for reviewing an IdP
>> and possibly blacklisting it.
>> > Because you are? responsible for clarin's pyff we have to work on this
>> together or rather you have to agree to it :).
>> >
>> > The VC or possibly the meeting at the dev conf. can help us defining
>> the baseline requirements. Besides that, we have this:
>> > 1. someone (here, the term someone is ok) finds suspicious IdP
>> > 2. creates ticket in TRAC
>> > 3. someone (here, the term someone should be better defined - tf-aai?)
>> reviews the requirements and comments whether any violations have been
>> found.
>> > 4. Andre closes the ticket, if there is a violation he will do it after
>> clarin's pyff is updated.
>> >
>> > One comment though, I do not think the xpath selection in this format
>> is sustainable in the long run. But it is up to you Andre to decide what to
>> do about it if anything.
>> >
>> > Best,
>> > Jozef
>> >
>> >
>> >
>> >
>> > On 1 May 2017 at 17:20, André Moreira <andre at clarin.eu> wrote:
>> > Hi Jozef,
>> >
>> >  My apologies for the late answer.
>> >  I must admit I am a bit out of context here; is your idea to
>> streamline a workflow for the detection and blacklisting of suspicious
>> IdPs, or just the later?
>> >
>> > >> 2. In 1a) or 1b) someone finds a suspicious IdP and CLARIN SPF
>> should be notified about it
>> > >>    QUESTION: what is the best/preferred way to notify CLARIN SPF -
>> trac? email?
>> >
>> >
>> > My preference is via a trac ticket, so we could have both: an immediate
>> notification sent by email, plus the status and history of the actions
>> taken.
>> > Furthermore, I think it is also important to keep an inventory of the
>> all the blacklisted IdPs and the respective reasoning for their
>> blacklisting. So I suggest we create a trac page for the 'IdPs blacklist
>> status’, but that the initial reports of misbehaving IdPs are done with the
>> creation of trac tickets. Then the operator handling one of these tickets,
>> would be responsible for updating the 'IdPs blacklist status’ page.
>> >
>> >
>> > >> 4. SPF will add this IdP into the filters
>> > >>   QUESTION: is filtering (blacklisting) of IdPs in CLARIN SPF
>> happening before they are published to https://infra.clarin.eu/aai/pr
>> od_md_about_spf_idps.xml and if so then where?
>> > >>   ACTION POINT: make filtered IdPs public (will be decided what is
>> the best way [1])
>> >
>> > Yes, filtering happens before publishing. You can see the PyFF code we
>> are using here: https://github.com/clarin-eric
>> /pyFF_config/blob/master/job_c.fd (see ’select’ XPath expressions).
>> >
>> >
>> > But again, I feel like I am not fully contextualised here, so I might
>> not be giving you the answers you were looking for. In that case, if it
>> suits you, we could schedule a skype call so we can have a more agile
>> discussion?
>> >
>> >
>> > Anyhow I hope this helps,
>> > Kind regards,
>> > ----
>> > André Moreira
>> > CLARIN ERIC
>> > https://www.clarin.eu
>> >
>> >
>> >
>> > > On 18 Apr 2017, at 12:29, Jozef Misutka <misutka at ufal.mff.cuni.cz>
>> wrote:
>> > >
>> > > Hi Andre,
>> > >
>> > > (cc-ing tf-aai@ too for comments)
>> > >
>> > > we should have a more formal and public way how to blacklist IdPs
>> from CLARIN SPF. We have the tools, so we really need only minor changes
>> and better documentation. In reality, this will be very rare but should be
>> documented nevertheless.
>> > >
>> > > The flow is like this:
>> > >
>> > > 1. User authenticates to one of CLARIN SPF SPs
>> > >    a) we have information about the IdP in
>> https://lindat.mff.cuni.cz/services/aaggreg/ so we can check if it looks
>> suspicious or not;
>> > >    b) different web applications inform about new users (e.g.,
>> clarin-dspace sends an email) so we can even check if the attributes seem
>> ok.
>> > >    ACTION POINT (optional): email new IdPs from attribute aggregator
>> > >
>> > > 2. In 1a) or 1b) someone finds a suspicious IdP and CLARIN SPF should
>> be notified about it
>> > >    QUESTION: what is the best/preferred way to notify CLARIN SPF -
>> trac? email?
>> > >
>> > > 3. CLARIN evaluates if that IdP is really problematic or not
>> > >    ACTION POINT: define a problematic/non problematic IdP e.g.,
>> problematic IdP is the one that does not validate its users more that by
>> sending emails
>> > >    ACTION POINT: make this definition public (will be decided what is
>> the best way [1])
>> > >
>> > > 4. SPF will add this IdP into the filters
>> > >   QUESTION: is filtering (blacklisting) of IdPs in CLARIN SPF
>> happening before they are published to https://infra.clarin.eu/aai/pr
>> od_md_about_spf_idps.xml and if so then where?
>> > >   ACTION POINT: make filtered IdPs public (will be decided what is
>> the best way [1])
>> > >
>> > >
>> > > To all, please feel free to comment, share practices etc. At LINDAT,
>> we have the following page
>> > > https://lindat.mff.cuni.cz/en/how-do-i-sign-up (see section
>> Supported Organisations)
>> > >
>> > > Best,
>> > > Jozef
>> > >
>> > >
>> > > [1] In my todo list is still to prepare a short CLARIN AAI
>> requirements document where we could add it and publish that document.
>> >
>> >
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clarin.eu/cgi-bin/mailman/private/tf-aai/attachments/20170525/65b12ea0/attachment.htm>