Page Comparison

...

[14:39:23 CST(-0600)] <EricDalquist> drewwills: you around?

[14:57:48 CST(-0600)] <drewwills> yep

[14:57:55 CST(-0600)] <EricDalquist> so more PD stuff

[14:58:06 CST(-0600)] <EricDalquist> do you ever see the need for more complex person queries?

[14:58:07 CST(-0600)] <drewwills> ok

[14:58:18 CST(-0600)] <EricDalquist> right now we have this terrible API

[14:58:27 CST(-0600)] <drewwills> +1

[14:58:29 CST(-0600)] <EricDalquist> where you give it a map of attribute name>value pairs

[14:58:40 CST(-0600)] <EricDalquist> and each source figures out how it will handle those

[14:58:49 CST(-0600)] <EricDalquist> do we really need something more expressive?

[14:59:15 CST(-0600)] <drewwills> maybe... can you throw out an example?

[14:59:17 CST(-0600)] <EricDalquist> like something where you could do (firstName=Jane && (lastName=Doe || lastName=Smith))

[14:59:33 CST(-0600)] <drewwills> hmmm... ppossibly

[14:59:53 CST(-0600)] <drewwills> that would suppost full-featured directory searching, for example

[15:00:04 CST(-0600)] <EricDalquist> note that something like that really only works for attribute sources that support a more free-form query building

[15:00:18 CST(-0600)] <EricDalquist> sources that are using direct named attribute replacement wouldn't really fit here

[15:00:22 CST(-0600)] <EricDalquist> they would either have to be ignored

[15:00:38 CST(-0600)] <EricDalquist> or queried once per result after getting results from the more flexible sources

[15:00:42 CST(-0600)] <drewwills> instead of passing attr/value pairs, you could pass criteria objects

[15:00:49 CST(-0600)] <EricDalquist> right

[15:01:18 CST(-0600)] <EricDalquist> I'd probably model something after: http://static.springsource.org/spring-ldap/site/apidocs/org/springframework/ldap/filter/package-frame.html

[15:01:34 CST(-0600)] <drewwills> looking

[15:02:28 CST(-0600)] <drewwills> sure... not dissimilar to PAGS and DLM evaluators actually

[15:02:35 CST(-0600)] <EricDalquist> yeah

[15:02:41 CST(-0600)] <EricDalquist> it is essentially the LDAP filter syntax

[15:02:42 CST(-0600)] <drewwills> i wonder if we could consolidate even

[15:02:45 CST(-0600)] <EricDalquist> in java object form

[15:02:58 CST(-0600)] <EricDalquist> so all of this is coming from a potential chunk of time I have from UW to do a big refactoring of PD

[15:03:09 CST(-0600)] <EricDalquist> our motivations are around better query support

[15:03:15 CST(-0600)] <EricDalquist> and better concurrent support

[15:03:28 CST(-0600)] <EricDalquist> we're starting to see login slowdowns due to the large number of sources

[15:04:10 CST(-0600)] <EricDalquist> I'll be transcribing this into the wiki once I have better design bits

[15:04:14 CST(-0600)] <EricDalquist> but here are my notes so far: https://gist.github.com/4262128

[15:04:20 CST(-0600)] <drewwills> hmmm... it's nice that we have WI at the bleeding edge to work out all the kinks

[15:04:25 CST(-0600)] <EricDalquist>

[15:04:36 CST(-0600)] <EricDalquist> nothing like 100k logins a day to make sure its working right

[15:04:45 CST(-0600)] <EricDalquist> the idea that we greatly simplify the attribute sources

[15:04:51 CST(-0600)] <EricDalquist> have a spring XML namespace for configuration

[15:05:01 CST(-0600)] <EricDalquist> and a big chunk of "core" code that does most of the heavy lifting

[15:05:13 CST(-0600)] <drewwills> yeah this would probably be a good match for a spring xsd

[15:05:18 CST(-0600)] <EricDalquist> I'm also hoping we can get rid of the explicit dao ordering and stuff

[15:05:56 CST(-0600)] <EricDalquist> just configure a buncvh of attribute sources

[15:06:05 CST(-0600)] <EricDalquist> optionally ordering and merge strategies

[15:06:34 CST(-0600)] <EricDalquist> and the core logic should be able to figure out which attribute queries it can run in parallel

[15:06:42 CST(-0600)] <EricDalquist> and which need to wait for others

[15:06:57 CST(-0600)] <drewwills> i help probably 4+ schools a year start a portal... persondirs is always a challenge

[15:07:08 CST(-0600)] <EricDalquist> yeah

[15:07:15 CST(-0600)] <EricDalquist> and that is why I'm bugging you for input

[15:07:23 CST(-0600)] <EricDalquist> pain points and ideas around them are great

[15:07:25 CST(-0600)] <drewwills> yeah

[15:07:30 CST(-0600)] <EricDalquist> to make sure we avoid/fix those issues

[15:07:44 CST(-0600)] <drewwills> here's one item of specific input...

[15:08:24 CST(-0600)] <drewwills> default for queryType should be OR imho

[15:08:32 CST(-0600)] <drewwills> or should have been i guess

[15:08:55 CST(-0600)] <drewwills> but that may not even be relevant in the updated api

[15:09:23 CST(-0600)] <EricDalquist> so if I query for (foo=x, bar=y) default to get results for (foo==x || bar ==y)

[15:09:40 CST(-0600)] <EricDalquist> ok

[15:09:44 CST(-0600)] <EricDalquist> so some concerns I have about the more complex query language

[15:09:49 CST(-0600)] <EricDalquist> what do we do when:

[15:10:11 CST(-0600)] <EricDalquist> query: (lastName == Smith && isStudent=Y)

[15:10:18 CST(-0600)] <EricDalquist> and we have two attribute sources

[15:10:30 CST(-0600)] <EricDalquist> one supports both lastName and isStudent attributes

[15:10:37 CST(-0600)] <EricDalquist> the other only supports lastName

[15:10:52 CST(-0600)] <drewwills> more often what i see is client code that take 1 search term and applies it to all the querry attrs

[15:10:59 CST(-0600)] <drewwills> user submits foo

[15:11:17 CST(-0600)] <drewwills> translates to username=foo, firstName=foo, lastName=foo

[15:11:27 CST(-0600)] <EricDalquist> and we can get much more complex queries and weird "which attribute sources answer)"

[15:11:35 CST(-0600)] <EricDalquist> ah

[15:11:36 CST(-0600)] <drewwills> it's better if the default is a hit if any one of those matches foo

[15:12:30 CST(-0600)] <EricDalquist> right

[15:12:47 CST(-0600)] <EricDalquist> so in that case running a query for (username=foo || firstName=foo || lastName=foo)

[15:12:54 CST(-0600)] <drewwills> exactly

[15:13:03 CST(-0600)] <EricDalquist> the thing i'm not sure about is what to do when you have multiple attribute sources

[15:13:29 CST(-0600)] <EricDalquist> and the incoming query references attributes that one of the attributes sources can't query on

[15:13:37 CST(-0600)] <EricDalquist> if we always do OR queries that isn't a problem

[15:13:55 CST(-0600)] <EricDalquist> but if we ever do ANDs it is

[15:14:12 CST(-0600)] <drewwills> yep, that makes sense

[15:15:04 CST(-0600)] <EricDalquist> and that is where I'm kind of stuck for ideas right now

[15:15:10 CST(-0600)] <drewwills> so perhaps the DAOs themselvs should not even have a queryType == AND | OR

[15:15:33 CST(-0600)] <drewwills> maybe we move to a criteria model, and let the client code handle that detail

[15:15:53 CST(-0600)] <EricDalquist> by a criteria model you mean like that LDAP filter API?

[15:16:25 CST(-0600)] <drewwills> yes... and instead it could have a loosy-goosey/strict setting

[15:16:41 CST(-0600)] <drewwills> as to whether it will forgive missing attrs

[15:16:51 CST(-0600)] <drewwills> default is probably strict

[15:17:14 CST(-0600)] <EricDalquist> so by default if the source doesnt support any attribute in the query

[15:17:18 CST(-0600)] <EricDalquist> that source isn't queried

[15:17:44 CST(-0600)] <drewwills> well maybe not

[15:17:46 CST(-0600)] <EricDalquist> part of me is thinking that I need to break this up more

[15:17:47 CST(-0600)] <EricDalquist> hrm

[15:17:49 CST(-0600)] <EricDalquist> oh!

[15:17:53 CST(-0600)] <EricDalquist> so here is an idea

[15:18:03 CST(-0600)] <drewwills> consider (username == 'foo' | lastName == 'bar')

[15:18:04 CST(-0600)] <EricDalquist> we break up "searchable sources" and "attribute sources"

[15:18:16 CST(-0600)] <drewwills> if you only support username, that's cool

[15:18:31 CST(-0600)] <drewwills> but if it's (username == 'foo' & lastName == 'bar')

[15:18:51 CST(-0600)] <drewwills> you must support both... unless (perhaps) strict=false

[15:19:03 CST(-0600)] <EricDalquist> ok

[15:19:28 CST(-0600)] <drewwills> i'm not convinced the strict thing is necessary... just a thought

[15:20:37 CST(-0600)] <EricDalquist> yeah

[15:21:09 CST(-0600)] <EricDalquist> I'm wondering if we really just need that search vs source seperation

[15:21:31 CST(-0600)]

 <EricDalquist> so attribute sources where you can configure a "query expression placeholder" the {0} that we have now

[15:21:35 CST(-0600)] <EricDalquist> we use those in searches

[15:21:50 CST(-0600)] <EricDalquist> and sources that use named parameters

[15:22:13 CST(-0600)] <EricDalquist> we don't treat as searchable

[15:22:13 CST(-0600)] <EricDalquist> hrm

[15:22:14 CST(-0600)] <EricDalquist> no

[15:22:18 CST(-0600)] <EricDalquist> that doesn't work either

[15:22:19 CST(-0600)] <EricDalquist> ok

[15:22:24 CST(-0600)] <EricDalquist> I'm going to go write up more examples

[15:37:14 CST(-0600)] <EricDalquist> so building on your strict/loose idea

[15:37:36 CST(-0600)] <EricDalquist> maybe "loose" does client side filtering

[15:37:43 CST(-0600)] <EricDalquist> so for a query of (firstName=Jane && (isStudent=Y || lastName=Doe))

[15:38:16 CST(-0600)] <EricDalquist> lets say we have sourceA that can query on all three attributes, we got ahead and run that query as is

[15:38:27 CST(-0600)] <EricDalquist> then we have sourceB that only supports firstName and lastName

[15:38:40 CST(-0600)] <EricDalquist> for B we run (firstName=Jane && (lastName=Doe))

[15:39:18 CST(-0600)] <EricDalquist> then during the merge process we apply that isStudent=Y filter in code