[Tf-aai] Filtering suspicious IdPs in CLARIN SPF

Mon May 29 09:03:50 CEST 2017

On 25 May 2017 at 09:57, Jozef Misutka <misutka at ufal.mff.cuni.cz> wrote:

>
>
> On 12 May 2017 at 10:24, Jozef Misutka <misutka at ufal.mff.cuni.cz> wrote:
>
>> And we have a case study from couple of hours ago,
>> https://trac.clarin.eu/ticket/1008#ticket
>>
>
> Anyone?
> Seems trac is not working as fast as (I would have) expected.
>
> Jozef
>
>
>
>>
>>
>> Steps 1. and 2. are done,
>> for 3. we do not have the requirements yet but anyway, please comment in
>> the trac ticket.
>>
>> 2Andre: (I am not trac admin) but I suggest you
>> a) create specific component in trac (AAI IdP Review?) and update the
>> tickets;
>> b) create the parallel trac page (I am not sure I know what you want
>> there) and include listing of all bugs that are open with specific
>> component on that page.
>>
>> Best,
>> Jozef
>>
>>
>> On 10 May 2017 at 17:53, André Moreira <andre at clarin.eu> wrote:
>>
>>> Hi Jozef,
>>>
>>> > And yes, I was talking only about the process flow for reviewing an
>>> IdP and possibly blacklisting it.
>>>
>>> Clear.
>>>
>>> > Because you are? responsible for clarin's pyff we have to work on this
>>> together or rather you have to agree to it :).
>>>
>>> Fair enough!
>>>
>>> > 1. someone (here, the term someone is ok) finds suspicious IdP
>>> > 2. creates ticket in TRAC
>>> > 3. someone (here, the term someone should be better defined - tf-aai?)
>>> reviews the requirements and comments whether any violations have been
>>> found.
>>> > 4. Andre closes the ticket, if there is a violation he will do it
>>> after clarin's pyff is updated.
>>>
>>>
>>> I still need the advantages of keeping a parallel trac page with the
>>> overall status of the blacklisting activities, especially since there is
>>> the potential for multiple people to handle step 3, this page can come
>>> handy. It is also a nice place to write the workflow we are discussing here
>>> in a wiki style.
>>> Thus I would modify 3 and 4:
>>> 3. someone (here, the term someone should be better defined - tf-aai?)
>>> reviews the requirements and comments whether any violations have been
>>> found updating the IdP blacklist status page on trac accordingly.
>>> 4. André closes the ticket and updates the blacklist status page
>>> accordingly, if there is a violation he will do it after clarin's pyff is
>>> updated.
>>>
>>> > One comment though, I do not think the xpath selection in this format
>>> is sustainable in the long run. But it is up to you Andre to decide what to
>>> do about it if anything.
>>>
>>> This has also been one of my remarks when I was introduced to the
>>> system, so I fully agree!
>>> My current idea is to create a small script that will convert a text
>>> file with all blacklisted IdPs (one per line) into the necessary XPaths and
>>> insert them into the job_c.fd code. I would like to hear if someone as an
>>> alternative idea.
>>>
>>> If more people are interested we can arrange a tf-aai meeting during the
>>> CLARIN centre meeting on Wednesday the 17th after lunch, since the
>>> afternoon is supposed to be dedicated to taskforce meetings anyway. Also
>>> feel free to grab me for an informal chat about this at anytime during the
>>> centre meeting.
>>>
>>>
>>> Regards,
>>> ----
>>> André Moreira
>>> CLARIN ERIC
>>> https://www.clarin.eu
>>>
>>>
>>>
>>> > On 2 May 2017, at 19:42, Jozef Misutka <misutka at ufal.mff.cuni.cz>
>>> wrote:
>>> >
>>> > Andre, you answered everything, thank you.
>>> >
>>> > And yes, I was talking only about the process flow for reviewing an
>>> IdP and possibly blacklisting it.
>>> > Because you are? responsible for clarin's pyff we have to work on this
>>> together or rather you have to agree to it :).
>>> >
>>> > The VC or possibly the meeting at the dev conf. can help us defining
>>> the baseline requirements. Besides that, we have this:
>>> > 1. someone (here, the term someone is ok) finds suspicious IdP
>>> > 2. creates ticket in TRAC
>>> > 3. someone (here, the term someone should be better defined - tf-aai?)
>>> reviews the requirements and comments whether any violations have been
>>> found.
>>> > 4. Andre closes the ticket, if there is a violation he will do it
>>> after clarin's pyff is updated.
>>> >
>>> > One comment though, I do not think the xpath selection in this format
>>> is sustainable in the long run. But it is up to you Andre to decide what to
>>> do about it if anything.
>>> >
>>> > Best,
>>> > Jozef
>>> >
>>> >
>>> >
>>> >
>>> > On 1 May 2017 at 17:20, André Moreira <andre at clarin.eu> wrote:
>>> > Hi Jozef,
>>> >
>>> >  My apologies for the late answer.
>>> >  I must admit I am a bit out of context here; is your idea to
>>> streamline a workflow for the detection and blacklisting of suspicious
>>> IdPs, or just the later?
>>> >
>>> > >> 2. In 1a) or 1b) someone finds a suspicious IdP and CLARIN SPF
>>> should be notified about it
>>> > >>    QUESTION: what is the best/preferred way to notify CLARIN SPF -
>>> trac? email?
>>> >
>>> >
>>> > My preference is via a trac ticket, so we could have both: an
>>> immediate notification sent by email, plus the status and history of the
>>> actions taken.
>>> > Furthermore, I think it is also important to keep an inventory of the
>>> all the blacklisted IdPs and the respective reasoning for their
>>> blacklisting. So I suggest we create a trac page for the 'IdPs blacklist
>>> status’, but that the initial reports of misbehaving IdPs are done with the
>>> creation of trac tickets. Then the operator handling one of these tickets,
>>> would be responsible for updating the 'IdPs blacklist status’ page.
>>> >
>>> >
>>> > >> 4. SPF will add this IdP into the filters
>>> > >>   QUESTION: is filtering (blacklisting) of IdPs in CLARIN SPF
>>> happening before they are published to https://infra.clarin.eu/aai/pr
>>> od_md_about_spf_idps.xml and if so then where?
>>> > >>   ACTION POINT: make filtered IdPs public (will be decided what is
>>> the best way [1])
>>> >
>>> > Yes, filtering happens before publishing. You can see the PyFF code we
>>> are using here: https://github.com/clarin-eric
>>> /pyFF_config/blob/master/job_c.fd (see ’select’ XPath expressions).
>>> >
>>> >
>>> > But again, I feel like I am not fully contextualised here, so I might
>>> not be giving you the answers you were looking for. In that case, if it
>>> suits you, we could schedule a skype call so we can have a more agile
>>> discussion?
>>> >
>>> >
>>> > Anyhow I hope this helps,
>>> > Kind regards,
>>> > ----
>>> > André Moreira
>>> > CLARIN ERIC
>>> > https://www.clarin.eu
>>> >
>>> >
>>> >
>>> > > On 18 Apr 2017, at 12:29, Jozef Misutka <misutka at ufal.mff.cuni.cz>
>>> wrote:
>>> > >
>>> > > Hi Andre,
>>> > >
>>> > > (cc-ing tf-aai@ too for comments)
>>> > >
>>> > > we should have a more formal and public way how to blacklist IdPs
>>> from CLARIN SPF. We have the tools, so we really need only minor changes
>>> and better documentation. In reality, this will be very rare but should be
>>> documented nevertheless.
>>> > >
>>> > > The flow is like this:
>>> > >
>>> > > 1. User authenticates to one of CLARIN SPF SPs
>>> > >    a) we have information about the IdP in
>>> https://lindat.mff.cuni.cz/services/aaggreg/ so we can check if it
>>> looks suspicious or not;
>>> > >    b) different web applications inform about new users (e.g.,
>>> clarin-dspace sends an email) so we can even check if the attributes seem
>>> ok.
>>> > >    ACTION POINT (optional): email new IdPs from attribute aggregator
>>>
>>
This has been implemented, write me if you want to receive these
notifications.

Date: 28 May 2017 at 19:05
Subject: New IdP [https://idp.ionio.gr/idp/shibboleth] used to access
CLARIN infrastructure
New IdP: [https://idp.ionio.gr/idp/shibboleth].
Check CLARIN attribute aggregator
<https://lindat.mff.cuni.cz/services/aaggreg/>.

Yours,
CLARIN attribute aggregator

Best,
Jozef

> > >
>>> > > 2. In 1a) or 1b) someone finds a suspicious IdP and CLARIN SPF
>>> should be notified about it
>>> > >    QUESTION: what is the best/preferred way to notify CLARIN SPF -
>>> trac? email?
>>> > >
>>> > > 3. CLARIN evaluates if that IdP is really problematic or not
>>> > >    ACTION POINT: define a problematic/non problematic IdP e.g.,
>>> problematic IdP is the one that does not validate its users more that by
>>> sending emails
>>> > >    ACTION POINT: make this definition public (will be decided what
>>> is the best way [1])
>>> > >
>>> > > 4. SPF will add this IdP into the filters
>>> > >   QUESTION: is filtering (blacklisting) of IdPs in CLARIN SPF
>>> happening before they are published to https://infra.clarin.eu/aai/pr
>>> od_md_about_spf_idps.xml and if so then where?
>>> > >   ACTION POINT: make filtered IdPs public (will be decided what is
>>> the best way [1])
>>> > >
>>> > >
>>> > > To all, please feel free to comment, share practices etc. At LINDAT,
>>> we have the following page
>>> > > https://lindat.mff.cuni.cz/en/how-do-i-sign-up (see section
>>> Supported Organisations)
>>> > >
>>> > > Best,
>>> > > Jozef
>>> > >
>>> > >
>>> > > [1] In my todo list is still to prepare a short CLARIN AAI
>>> requirements document where we could add it and publish that document.
>>> >
>>> >
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clarin.eu/cgi-bin/mailman/private/tf-aai/attachments/20170529/9da0d980/attachment.htm>