[Tf-aai] Filtering suspicious IdPs in CLARIN SPF

Sat Jun 10 20:11:04 CEST 2017

Dear all,

I commented on https://trac.clarin.eu/ticket/1008#comment:6
but I would like to distribute it also here.

That particular IdP is not part of SPF but before I created this issue I
checked using
https://met.refeds.org/met/entity/https%253A%252F%252Fidp.unicon.net%252Fidp%252Fshibboleth/
that it is part of one of CLARIN SPF members namely UK Federation.
Therefore, the process of SPF via eduGAIN should be re-visited because if
an entity is part of multiple federations, only one is chosen for eduGAIN.
If that particular one is not part of SPF, we *may* miss an IdP we want to
have in SPF.

Best,
Jozef

On 8 June 2017 at 17:35, André Moreira <andre at clarin.eu> wrote:

> Hi everybody,
>
> I have set up a page on the CLARIN trac describing the process of
> requesting changes to the AAI SPF blacklist. This page also keeps trac of
> the currently blacklisted IdPs, as well as ongoing blacklist requests
> (tickets). The page can be found here:
>
> https://trac.clarin.eu/wiki/ServiceProviderFederation/IdpBlacklist
>
> Please take a look when you have the opportunity. At this point any input
> is very welcome!
>
>
> Regards,
> ----
> André Moreira
> CLARIN ERIC
> https://www.clarin.eu
>
>
>
> > On 29 May 2017, at 11:27, André Moreira <andre at clarin.eu> wrote:
> >
> > Hi Jozef,
> >
> >> Anyone?
> >> Seems trac is not working as fast as (I would have) expected.
> >
> > My apologies but I really did not have the opportunity to look into this
> before. I am finishing some external assignments this month so it is
> difficult for me to handle all the SPF issues as quickly as I would like
> to. The good news is that starting next month, I will be working full time
> for CLARIN ERIC so I will definitely be able to handle trac issues in a
> more reasonable time frame.
> >
> > Anyway, I will try to look into this today or at most tomorrow!
> >
> >> This has been implemented, write me if you want to receive these
> notifications.
> >
> >
> > Yes, can you please add my address to receive them?
> >
> >
> > Kind regards,
> > ----
> > André Moreira
> > CLARIN ERIC
> > https://www.clarin.eu
> >
> >
> >
> >> On 25 May 2017, at 09:57, Jozef Misutka <misutka at ufal.mff.cuni.cz>
> wrote:
> >>
> >>
> >>
> >> On 12 May 2017 at 10:24, Jozef Misutka <misutka at ufal.mff.cuni.cz>
> wrote:
> >> And we have a case study from couple of hours ago,
> >> https://trac.clarin.eu/ticket/1008#ticket
> >>
> >> Anyone?
> >> Seems trac is not working as fast as (I would have) expected.
> >>
> >> Jozef
> >>
> >>
> >>
> >>
> >> Steps 1. and 2. are done,
> >> for 3. we do not have the requirements yet but anyway, please comment
> in the trac ticket.
> >>
> >> 2Andre: (I am not trac admin) but I suggest you
> >> a) create specific component in trac (AAI IdP Review?) and update the
> tickets;
> >> b) create the parallel trac page (I am not sure I know what you want
> there) and include listing of all bugs that are open with specific
> component on that page.
> >>
> >> Best,
> >> Jozef
> >>
> >>
> >> On 10 May 2017 at 17:53, André Moreira <andre at clarin.eu> wrote:
> >> Hi Jozef,
> >>
> >>> And yes, I was talking only about the process flow for reviewing an
> IdP and possibly blacklisting it.
> >>
> >> Clear.
> >>
> >>> Because you are? responsible for clarin's pyff we have to work on this
> together or rather you have to agree to it :).
> >>
> >> Fair enough!
> >>
> >>> 1. someone (here, the term someone is ok) finds suspicious IdP
> >>> 2. creates ticket in TRAC
> >>> 3. someone (here, the term someone should be better defined - tf-aai?)
> reviews the requirements and comments whether any violations have been
> found.
> >>> 4. Andre closes the ticket, if there is a violation he will do it
> after clarin's pyff is updated.
> >>
> >>
> >> I still need the advantages of keeping a parallel trac page with the
> overall status of the blacklisting activities, especially since there is
> the potential for multiple people to handle step 3, this page can come
> handy. It is also a nice place to write the workflow we are discussing here
> in a wiki style.
> >> Thus I would modify 3 and 4:
> >> 3. someone (here, the term someone should be better defined - tf-aai?)
> reviews the requirements and comments whether any violations have been
> found updating the IdP blacklist status page on trac accordingly.
> >> 4. André closes the ticket and updates the blacklist status page
> accordingly, if there is a violation he will do it after clarin's pyff is
> updated.
> >>
> >>> One comment though, I do not think the xpath selection in this format
> is sustainable in the long run. But it is up to you Andre to decide what to
> do about it if anything.
> >>
> >> This has also been one of my remarks when I was introduced to the
> system, so I fully agree!
> >> My current idea is to create a small script that will convert a text
> file with all blacklisted IdPs (one per line) into the necessary XPaths and
> insert them into the job_c.fd code. I would like to hear if someone as an
> alternative idea.
> >>
> >> If more people are interested we can arrange a tf-aai meeting during
> the CLARIN centre meeting on Wednesday the 17th after lunch, since the
> afternoon is supposed to be dedicated to taskforce meetings anyway. Also
> feel free to grab me for an informal chat about this at anytime during the
> centre meeting.
> >>
> >>
> >> Regards,
> >> ----
> >> André Moreira
> >> CLARIN ERIC
> >> https://www.clarin.eu
> >>
> >>
> >>
> >>> On 2 May 2017, at 19:42, Jozef Misutka <misutka at ufal.mff.cuni.cz>
> wrote:
> >>>
> >>> Andre, you answered everything, thank you.
> >>>
> >>> And yes, I was talking only about the process flow for reviewing an
> IdP and possibly blacklisting it.
> >>> Because you are? responsible for clarin's pyff we have to work on this
> together or rather you have to agree to it :).
> >>>
> >>> The VC or possibly the meeting at the dev conf. can help us defining
> the baseline requirements. Besides that, we have this:
> >>> 1. someone (here, the term someone is ok) finds suspicious IdP
> >>> 2. creates ticket in TRAC
> >>> 3. someone (here, the term someone should be better defined - tf-aai?)
> reviews the requirements and comments whether any violations have been
> found.
> >>> 4. Andre closes the ticket, if there is a violation he will do it
> after clarin's pyff is updated.
> >>>
> >>> One comment though, I do not think the xpath selection in this format
> is sustainable in the long run. But it is up to you Andre to decide what to
> do about it if anything.
> >>>
> >>> Best,
> >>> Jozef
> >>>
> >>>
> >>>
> >>>
> >>> On 1 May 2017 at 17:20, André Moreira <andre at clarin.eu> wrote:
> >>> Hi Jozef,
> >>>
> >>> My apologies for the late answer.
> >>> I must admit I am a bit out of context here; is your idea to
> streamline a workflow for the detection and blacklisting of suspicious
> IdPs, or just the later?
> >>>
> >>>>> 2. In 1a) or 1b) someone finds a suspicious IdP and CLARIN SPF
> should be notified about it
> >>>>>   QUESTION: what is the best/preferred way to notify CLARIN SPF -
> trac? email?
> >>>
> >>>
> >>> My preference is via a trac ticket, so we could have both: an
> immediate notification sent by email, plus the status and history of the
> actions taken.
> >>> Furthermore, I think it is also important to keep an inventory of the
> all the blacklisted IdPs and the respective reasoning for their
> blacklisting. So I suggest we create a trac page for the 'IdPs blacklist
> status’, but that the initial reports of misbehaving IdPs are done with the
> creation of trac tickets. Then the operator handling one of these tickets,
> would be responsible for updating the 'IdPs blacklist status’ page.
> >>>
> >>>
> >>>>> 4. SPF will add this IdP into the filters
> >>>>>  QUESTION: is filtering (blacklisting) of IdPs in CLARIN SPF
> happening before they are published to https://infra.clarin.eu/aai/
> prod_md_about_spf_idps.xml and if so then where?
> >>>>>  ACTION POINT: make filtered IdPs public (will be decided what is
> the best way [1])
> >>>
> >>> Yes, filtering happens before publishing. You can see the PyFF code we
> are using here: https://github.com/clarin-eric/pyFF_config/blob/master/
> job_c.fd (see ’select’ XPath expressions).
> >>>
> >>>
> >>> But again, I feel like I am not fully contextualised here, so I might
> not be giving you the answers you were looking for. In that case, if it
> suits you, we could schedule a skype call so we can have a more agile
> discussion?
> >>>
> >>>
> >>> Anyhow I hope this helps,
> >>> Kind regards,
> >>> ----
> >>> André Moreira
> >>> CLARIN ERIC
> >>> https://www.clarin.eu
> >>>
> >>>
> >>>
> >>>> On 18 Apr 2017, at 12:29, Jozef Misutka <misutka at ufal.mff.cuni.cz>
> wrote:
> >>>>
> >>>> Hi Andre,
> >>>>
> >>>> (cc-ing tf-aai@ too for comments)
> >>>>
> >>>> we should have a more formal and public way how to blacklist IdPs
> from CLARIN SPF. We have the tools, so we really need only minor changes
> and better documentation. In reality, this will be very rare but should be
> documented nevertheless.
> >>>>
> >>>> The flow is like this:
> >>>>
> >>>> 1. User authenticates to one of CLARIN SPF SPs
> >>>>   a) we have information about the IdP in https://lindat.mff.cuni.cz/
> services/aaggreg/ so we can check if it looks suspicious or not;
> >>>>   b) different web applications inform about new users (e.g.,
> clarin-dspace sends an email) so we can even check if the attributes seem
> ok.
> >>>>   ACTION POINT (optional): email new IdPs from attribute aggregator
> >>>>
> >>>> 2. In 1a) or 1b) someone finds a suspicious IdP and CLARIN SPF should
> be notified about it
> >>>>   QUESTION: what is the best/preferred way to notify CLARIN SPF -
> trac? email?
> >>>>
> >>>> 3. CLARIN evaluates if that IdP is really problematic or not
> >>>>   ACTION POINT: define a problematic/non problematic IdP e.g.,
> problematic IdP is the one that does not validate its users more that by
> sending emails
> >>>>   ACTION POINT: make this definition public (will be decided what is
> the best way [1])
> >>>>
> >>>> 4. SPF will add this IdP into the filters
> >>>>  QUESTION: is filtering (blacklisting) of IdPs in CLARIN SPF
> happening before they are published to https://infra.clarin.eu/aai/
> prod_md_about_spf_idps.xml and if so then where?
> >>>>  ACTION POINT: make filtered IdPs public (will be decided what is the
> best way [1])
> >>>>
> >>>>
> >>>> To all, please feel free to comment, share practices etc. At LINDAT,
> we have the following page
> >>>> https://lindat.mff.cuni.cz/en/how-do-i-sign-up (see section
> Supported Organisations)
> >>>>
> >>>> Best,
> >>>> Jozef
> >>>>
> >>>>
> >>>> [1] In my todo list is still to prepare a short CLARIN AAI
> requirements document where we could add it and publish that document.
> >>>
> >>>
> >>
> >>
> >>
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clarin.eu/cgi-bin/mailman/private/tf-aai/attachments/20170610/73e8935e/attachment.htm>