uPortal IRC Logs-2007-11-02

Owned by Apereo Infrastructure
Last updated: Nov 02, 2007Version comment
6 min read

[00:01:09 EDT(-0400)] * esm (n=esm@207-53-193-26.dynamic-dsl.qis.net) has joined ##uportal
[00:21:42 EDT(-0400)] * esm (n=esm@207-53-193-26.dynamic-dsl.qis.net) has joined ##uportal
[08:18:22 EDT(-0400)] * esm (n=esm@207-53-193-26.dynamic-dsl.qis.net) has joined ##uportal
[09:25:35 EDT(-0400)] * jayshao (n=jayshao@130.156.176.102) has joined ##uportal
[09:36:00 EDT(-0400)] * awills (n=awills@ip72-201-156-173.ph.ph.cox.net) has joined ##uportal
[09:41:36 EDT(-0400)] * awills (n=awills@ip72-201-156-173.ph.ph.cox.net) has left ##uportal
[09:58:23 EDT(-0400)] * EricDalquist (n=dalquist@bohemia.doit.wisc.edu) has joined ##uportal
[10:05:23 EDT(-0400)] * michelled (n=team@142.150.154.114) has joined ##uportal
[10:37:59 EDT(-0400)] * colinclark (n=atrcwrk2@142.150.154.101) has joined ##uportal
[10:40:23 EDT(-0400)] * colinclark (n=atrcwrk2@142.150.154.101) has left ##uportal
[10:40:42 EDT(-0400)] * colinclark (n=atrcwrk2@142.150.154.101) has joined ##uportal
[10:57:29 EDT(-0400)] * pberry (n=pberry@waldorf.CSUChico.EDU) has joined ##uportal
[11:01:44 EDT(-0400)] <deuce_> hey eric, i just checked in the changes for moving the bootstrap mvn dependencies to the repo (http://www.ja-sig.org/issues/browse/UP-1860)
[11:02:03 EDT(-0400)] <EricDalquist> awesome, thanks
[11:02:32 EDT(-0400)] <deuce_> np
[11:05:58 EDT(-0400)] * awills (n=awills@12.164.136.183) has joined ##uportal
[11:12:16 EDT(-0400)] * athena7 (n=athena@c-24-2-130-130.hsd1.ct.comcast.net) has joined ##uportal
[11:12:23 EDT(-0400)] <EricDalquist> deuce_: could you also update http://www.ja-sig.org/wiki/display/UPC/Building+and+Deploying+uPortal+3 and remove the bootstrap refs?
[11:12:30 EDT(-0400)] <deuce_> surely
[11:12:36 EDT(-0400)] <EricDalquist> thanks
[11:27:36 EDT(-0400)] <deuce_> eric .. is ant initportal necessary for running maven goals now that we've removed the bootstrapping step?
[11:27:45 EDT(-0400)] <EricDalquist> no
[11:27:52 EDT(-0400)] <deuce_> cool thx
[11:46:25 EDT(-0400)] * jayshao (n=jayshao@130.156.176.102) has joined ##uportal
[12:23:49 EDT(-0400)] * jayshao (n=jayshao@130.156.176.102) has joined ##uportal
[12:42:55 EDT(-0400)] <esm> so i got mad enough at svn_load_dirs.pl that i'm writing something to help.
[12:43:04 EDT(-0400)] <EricDalquist> (smile)
[12:43:05 EDT(-0400)] <esm> manage vendor branches
[12:43:13 EDT(-0400)] <EricDalquist> how are you going to do it?
[12:43:22 EDT(-0400)] <EricDalquist> I was thinking about this problem on the bus the other day too
[12:43:32 EDT(-0400)] <esm> yeah. feed back is welcome so here goes
[12:44:10 EDT(-0400)] <esm> compare directory structures of vendor drop version X and version Y.
[12:44:33 EDT(-0400)] <esm> (taking a set of ignore files)
[12:45:28 EDT(-0400)] <esm> test to see if files in version X exist at the same location in version Y.
[12:46:00 EDT(-0400)] <esm> if so, then no problem.
[12:46:53 EDT(-0400)] <esm> otherwise the file is new in version Y or removed in version X, OR it has been moved.
[12:47:54 EDT(-0400)] <esm> so at this point we have candidates for addition and candidates for removal.
[12:48:23 EDT(-0400)] <esm> but some of those candidates may be really moves.
[12:49:01 EDT(-0400)] <esm> so I create an index of addtions and removals by basename
[12:50:10 EDT(-0400)] <esm> then do a byte by byte comparison for each. if they aren't byte for byte the same file, then I use a distance algorithm to determine a probablility that they are equivilant files.
[12:50:27 EDT(-0400)] <esm> so in my hacked out code i have something like
[12:52:05 EDT(-0400)] <esm> http://rafb.net/p/EZHUBF81.html
[12:52:23 EDT(-0400)] <EricDalquist> fun
[12:52:24 EDT(-0400)] <EricDalquist> so
[12:52:48 EDT(-0400)] <esm> at this juncture i'm not even doing anything with SVN. Its just a directory compare.
[12:52:57 EDT(-0400)] <EricDalquist> sounds sane
[12:53:03 EDT(-0400)] <EricDalquist> but why not have a vendor-branch helper script that actually runs against the vendor's SVN repository?
[12:53:23 EDT(-0400)] <EricDalquist> looks at the log from V1 to V2, tracks all of the moves
[12:53:29 EDT(-0400)] <EricDalquist> and uses that info to help with the merge
[12:53:49 EDT(-0400)] <esm> yeah one of my colleages suggested that as well.
[12:54:04 EDT(-0400)] <esm> since the vendor repo actually has all this history
[12:54:10 EDT(-0400)] <EricDalquist> yeah
[12:54:24 EDT(-0400)] <EricDalquist> it would be nice to have vendor-export and vendor-import scripts
[12:54:35 EDT(-0400)] <EricDalquist> the -export would be given two revs or some such
[12:54:52 EDT(-0400)] <EricDalquist> and pretty much do a checkout + track all moves
[12:55:15 EDT(-0400)] <EricDalquist> then the -import could 'replay' the moves on your local vendor branch before dumping the new files in
[12:56:02 EDT(-0400)] <esm> thats a good idea. yeah this thing I'm writing will basically output a replay and be able to read in the replay.
[12:56:29 EDT(-0400)] <esm> To be honest the SvnKit java library looks daunting. Thats one reason why.
[12:56:50 EDT(-0400)] <esm> but i guess the advantage is you have 100% confidence what is a move and what isn't.
[12:56:52 EDT(-0400)] <EricDalquist> the only problem with my suggestion is if the source for your vendor-branch isn't a svn repo as well
[12:56:55 EDT(-0400)] <EricDalquist> yeah
[12:57:12 EDT(-0400)] <EricDalquist> I was thinking ruby or perl & regex on command line calls
[12:57:26 EDT(-0400)] <EricDalquist> pretty much do a 'svn -v log' and parse the output (tongue)
[12:57:43 EDT(-0400)] <esm> oh. Yeah, I would probably use SvnKit, since its pretty baked.
[12:58:38 EDT(-0400)] <esm> thanks for the thoughts... so I'm wondering if I should change my approach
[12:58:43 EDT(-0400)] <EricDalquist> yup (smile)
[12:58:54 EDT(-0400)] <awills> for what it's worth, gentlemen, this is the sort of thing you could hit pretty hard w/ cernunnos
[12:59:15 EDT(-0400)] <EricDalquist> if you know enough perl to do it there you could probably get the final result included with svnmerge and the rest of those tools
[12:59:37 EDT(-0400)] <esm> oh there's no way I'm ever touching perl (smile)
[12:59:54 EDT(-0400)] <esm> if it takes the same arguments as svn_load_dirs.pl I bet it would be accepted as a contrib project
[12:59:59 EDT(-0400)] <awills> i've done some SVN discovery, and plenty of file comparison/manipulation
[13:00:13 EDT(-0400)] <esm> awills: interesting
[13:00:23 EDT(-0400)] <awills> sec...
[13:00:36 EDT(-0400)] <EricDalquist> lol
[13:01:00 EDT(-0400)] <awills> http://cernunnos.googlecode.com/svn/etc/Portlet/util/fetch-dir.crn
[13:01:12 EDT(-0400)] <esm> http://svnkit.com/kb/javadoc/index.html
[13:01:16 EDT(-0400)] <esm> er
[13:01:29 EDT(-0400)] <esm> http://svnkit.com/kb/javadoc/org/tmatesoft/svn/core/replicator/ISVNReplicationHandler.html
[13:01:31 EDT(-0400)] <awills> uses nekohtml to discover the contents of an SVN path over http
[13:01:35 EDT(-0400)] <esm> ok this may be easier than i thought.
[13:02:04 EDT(-0400)] <EricDalquist> awills: you'd need to be able to run the svn log -v command or something similar
[13:02:09 EDT(-0400)] <EricDalquist> which isn't available over plain http
[13:02:41 EDT(-0400)] <esm> ok yeah this should be not bad...
[13:02:49 EDT(-0400)] <EricDalquist> for this application you need the full log history for a path from rev X to rev Y and then parse out the moves
[13:02:52 EDT(-0400)] <EricDalquist> (smile)
[13:03:05 EDT(-0400)] <EricDalquist> esm: and on the -import side you should be able to replay all of the moves locally
[13:03:09 EDT(-0400)] <EricDalquist> then do the overlay
[13:03:10 EDT(-0400)] <esm> EricDalquist: oh
[13:03:20 EDT(-0400)] <EricDalquist> so it is atomic from the repos point of view
[13:03:43 EDT(-0400)] <awills> oh very true EricDalquist, but using neko for part doesn't prevent you from using something else for another part... and you'd be able to crank out atleast some features almost immediately
[13:04:12 EDT(-0400)] <EricDalquist> heh, except reading the contents of the svn repo via http isn't needed at all
[13:04:29 EDT(-0400)] <EricDalquist> you'd either have to write hooks int SVNKit or do shell calls to the svn command
[13:04:54 EDT(-0400)] <EricDalquist> the http part of svn, while handy, is rather limited since all you can ever see is HEAD
[13:04:59 EDT(-0400)] <awills> either is fine (smile) and if you change your mind later, it's pretty easy to switch
[13:06:48 EDT(-0400)] <esm> ok nice. yes yes
[13:06:57 EDT(-0400)] <esm> i'll bite the bullet and dig into svnkit
[13:07:01 EDT(-0400)] <EricDalquist> (smile)
[13:07:06 EDT(-0400)] <EricDalquist> let me know how it goes
[13:07:12 EDT(-0400)] <esm> i'll probably get done in an afternoon what a day has already cost me
[13:07:14 EDT(-0400)] <EricDalquist> sounds like it should be a pretty useful tool
[13:07:15 EDT(-0400)] <esm> which is good i guess.
[13:07:48 EDT(-0400)] <esm> yeah thanks for the input, much appreciated
[13:08:06 EDT(-0400)] <EricDalquist> yup
[13:08:32 EDT(-0400)] <EricDalquist> you know
[13:08:34 EDT(-0400)] <EricDalquist> thinking about it more
[13:08:52 EDT(-0400)] <EricDalquist> what you really just need is a client way of doing 'svnadmin dump X Y'
[13:09:03 EDT(-0400)] <EricDalquist> then load it into your vendorbranch
[13:09:17 EDT(-0400)] <EricDalquist> now that wouldn't work because the dump file doesn't play nice with changing paths
[13:09:25 EDT(-0400)] <EricDalquist> but that concept is pretty much it
[13:09:33 EDT(-0400)] <esm> 1.4 + has svnsync which allows history replay
[13:09:43 EDT(-0400)] <esm> i think it is svnsync i forget.
[13:09:44 EDT(-0400)] <EricDalquist> it would be great when doing SVN vendor branches to get the vendor's history too
[13:10:24 EDT(-0400)] <esm> http://svnkit.com/kb/javadoc/org/tmatesoft/svn/core/replicator/SVNRepositoryReplicator.html
[13:11:11 EDT(-0400)] <EricDalquist> hrm
[13:11:16 EDT(-0400)] <EricDalquist> I wonder if that would let you change the paths
[13:11:47 EDT(-0400)] <EricDalquist> you know replicate server.com/foo/product/branch to mine.com/vendor/product or some such
[13:11:47 EDT(-0400)] <esm> dunno but i'm firing up some tests now
[13:11:51 EDT(-0400)] <EricDalquist> (smile)
[13:11:54 EDT(-0400)] <esm> right
[13:14:22 EDT(-0400)] <esm> bbiab
[13:34:07 EDT(-0400)] * andrew_petro_ubu (n=apetro@uni1.unicon.net) has joined ##uportal
[14:11:04 EDT(-0400)] * jayshao (n=jayshao@130.156.176.102) has joined ##uportal
[14:16:15 EDT(-0400)] * colinclark (n=atrcwrk2@142.150.154.101) has joined ##uportal
[15:12:33 EDT(-0400)] * ac_chan (n=alex@tempoutsidepix.pratt.edu) has joined ##uportal
[15:22:14 EDT(-0400)] <EricDalquist> how's the svn stuff going esm?
[15:22:37 EDT(-0400)] <esm> going ok
[15:22:49 EDT(-0400)] <esm> just trying to figure out the best approach with svnkit
[15:23:09 EDT(-0400)] <esm> i don't want to re-invent the wheel...
[15:23:37 EDT(-0400)] <EricDalquist> (smile)
[15:23:51 EDT(-0400)] <esm> like part of the problem is that svn merge doesen't operate across repositories
[15:23:59 EDT(-0400)] <esm> at least that was true in the 1.3.x world
[15:24:14 EDT(-0400)] <esm> so i'm just double-checking some assumptions
[15:26:04 EDT(-0400)] <esm> it still looks like in 1.4 merge operates on a single repository
[16:18:21 EDT(-0400)] * ac_chan (n=alex@tempoutsidepix.pratt.edu) has left ##uportal
[16:34:35 EDT(-0400)] <pberry> have a nice weekend, everybody
[16:38:05 EDT(-0400)] * michelled (n=team@142.150.154.114) has left ##uportal
[16:53:38 EDT(-0400)] <esm> hmm i don't know if I can get the "copied from" piece
[16:54:14 EDT(-0400)] <esm> oh wait mebby i can..
[16:58:09 EDT(-0400)] <esm> http://rafb.net/p/D4iCn318.html
[16:58:43 EDT(-0400)] <esm> ok that was relatively painless...
[17:03:23 EDT(-0400)] <esm> http://rafb.net/p/IjlFw450.html
[18:10:14 EDT(-0400)] * deuce (n=deuce@ip68-3-31-85.ph.ph.cox.net) has joined ##uportal
[19:03:29 EDT(-0400)] * colinclark (n=atrcwrk2@142.150.154.101) has left ##uportal
[20:28:58 EDT(-0400)] * andrew_petro_ubu (n=apetro@uni1.unicon.net) has left ##uportal
[21:05:23 EDT(-0400)] * awills (n=awills@12.164.136.183) has left ##uportal