Frank's Activity Log

Admin

• LDAP check for error alert went off... turns out the attribute being checked to determine that an entry is in the "Expired" state was misspelled. Corrected.
• mink ssh closed to off-campus access after someone in Japan thought it would be nice to try to break in yesterday
• Working on OpenLDAP 2.3 builds for RHEL3 and RHEL4
• sendpage locked up... nasty process to fix
• Reset user's password (cover for sick Account Services reps)
• Updated a calendar user's email address (cover for sick Account Services rep)
• Created a calendar account...

Posted by at 8:09 AM | Permalink | Comments (0)

Admin

• Created daily job to search for the LDAP problem that got me paged at 3am today. I don't know how it was created, so I need to catch it when it happens -- assuming it is part of the daily update -- to see what is causing it
• Re-run the Active Directory update now that the LDAP update has been fixed

On-Call

• No pages

Posted by at 8:01 AM | Permalink | Comments (0)

Admin

• run "diag" on H70 that is still complaining from the power outage of the 19th -- it finds nothing... so what is the system finding to complain about??? --- H70 service/install guide says P1-M1 is memory module 1 on planar 1, so pluggable modules 7 and 8 are in trouble
• upgrade ldap.uvm.edu servers and convert to new vips stuff
  • peregrine grub fails...
  • Had to boot into "rescue" mode and run 'grub-install /dev/sda'

On Call

• chipmunk cpu -- dynaweb looping again
• LDAP Update failed -- uvmexppidm multiple values caused by bad logic in update perl script

Posted by at 9:26 AM | Permalink | Comments (0)

Admin

• Showed up...that was an achievement in itself this morning!
• RHN required maintenance applied to systems
• Rebooted the master ldap server... kernel updated
• Wiki created for ldap.uvm.edu documentation
• Working with Jim on new kernel 2.6 arguments that may remove the need for the custom kernels we've been running on RHEL4 machines
  • It Works!
• Experimenting more with sync-repl replication for LDAP

Posted by at 7:50 AM | Permalink | Comments (0)

Admin

• LDAP Student Employee error -- found student and connected the two entries
• TRU: 64851685/64851690 -- battle continues... SUN Silver Support is unreliable!
  • Apparently, the controller that came in last night was the wrong style. The correct style (new style) should arrive today at noon. Then we can see if that fixes it or we still have other problems to find and repair
  • Apparently, Mike and I are both good at stirring up hornet's nests... NIS (the people that sold us the support) are asking for a conference call to discuss what happened and how we can be assured that it will never happen again -- based on a note I sent to the sales person that we were unhappy with SUN's response
  • Sun engineer and Sun SE arrived with the correct controller and the hardware came back up nice as could be.
  • Oracle folks are now working to get the databases back into a decent state and get the application back up.
• Update ph2ldap configuration so it doesn't return the internal routing address which will only confuse people

Posted by at 8:06 AM | Permalink | Comments (0)

Admin

• TRU:
  • SUN Hardware case 64851690
  • Easter-Egg HBA from cottonmouth ... no change - A1000 still not talking
  • Easter-Egg cable from POD ... No change...
  • SUN worked through it with me for 90 minutes and is dispatching an engineer with a replacement controller for the A1000.
• caltest: restart network... eth0 didn't start yesterday
• FROG:
  • SUN Hardware case 64851675 - HDisk failed in D1000.
  • SSHD not working correctly... still unsure what is broken
  • SSHD problem turned out to be that the old frog was actively stealing the IP address

Posted by at 6:51 AM | Permalink | Comments (0)

Admin

• On Call for tonight
• Have SecurID servers put subjects on their nightly backup script emails
• UPS failure takes down Waterman machine room...

Posted by at 8:37 AM | Permalink | Comments (0)

Admin

• looping httpd's on one of the web servers drove load over 7, using "prstat -s cpu -n 10" found the loopers and killed them off
• Correct a problem with the feed from OpenLDAP into Active Directory that was caused by the FootPrints entry being recreated yesterday
• Reset forgotten password for a blog

LDAP

• Build test replica server (mink) using 2.2.30 RPMs
• Successfully configure slurpd replication on it
• Start working on syncrepl replication

Posted by at 9:13 AM | Permalink | Comments (0)

LDAP/Accounts

• Working on the conversion scripts to move ou=People,ou=Expired and ou=Aliases into ou=People tree branch
  1. duplicate footprints dn in ou=People
  2. update slapd.conf on all servers with new dn
  3. recycle all ldap servers
     • Oops! DO NOT USE slapd_db_recover... it messed up porcupine!
  4. update footprints config to use new dn
  5. run conversion script
  6. remove old footprints dn in ou=People,ou=Expired
     • Oops... footprints has the DN in two places
     • removing the old DN caused logins to fail for 9 minutes (from 10:47 to 10:56)
  7. update slapd.conf on all servers to remove old dn
  8. recycle all ldap servers
  • Started at 9 AM
  • Completed at 11:50 AM
  • Caused two problems
    • porcupine's database corrupted by running slapd_db_recover
    • footprints unable to log people in for 9 minutes
• Finally got OpenLDAP 2.2.30 to build on RHEL3

Team

• meeting with Network Services
• Team meeting -- mission statement

Posted by at 8:32 AM | Permalink | Comments (0)

Admin

• Account Services wants an account provisioned in AD... this is getting to be far too frequent a request to be handled in the one-off fashion we have been doing... grrr
  • Turns out the person's paperwork has not even made it to HR yet
  • Perhaps CIT needs to talk about how realistic it is to create accounts for people ahead of time anymore
• Oracle (TAR 4972058.992) says the db_VISTA ERROR 1 was caused by my aborted unirestore on Monday. No problem!!! YAY!!

Posted by at 8:12 AM | Permalink | Comments (0)

Admin

• One more round of Oracle wanting actions that don't make a lot of sense. We've worked around the problem with not being able to install the v10 client and now they want us to go and install the v9 client as the domain user and then upgrade to the v10 client to see if it is a bug in the installer or not. I've asked Lou if he wants to do this or not.
• Lou attempted the Oracle action plan and it failed to even be able to install the v9.0.4.2 client. I've updated the TAR with the failure information
• H50 power supplies... are they hot swappable? Yes, they are -- as long as you have two of them... Swapped a good one from gnu into okapi
• Recovered calendar information for the Dean of College of Education

Calendar

• TAR 4972058.992 created
  • db_VISTA ERROR 1 -> SetPositionFirst: d_findfm
  • Happened at 13:45 yesterday
  • Uploaded the eng.log and dbv.log files per Oracle request

Posted by at 8:33 AM | Permalink | Comments (0)

Admin

• recover ldap database on penguinx. It apparently got corrupted at some point and needed a recover to fix it. Penguinx had hung during the night and been power cycled. An ldapsearch had hung on the BB monitoring system, and slurpd was unable to feed updates to it from before 2am.
• Continue work on LDAP code... removal of most of ou=Aliases and ou=People,ou=Expired branches
• Look up the SyncML requirement for Axim X5 again to prove it to another person that the PALM Sync software is NOT for use with the PPC products
• Make a symbolic link on the banner system
• Add samba password for account on banner system
• Add samba password for accounts on development system
• Restore calendar data for Carol

Posted by at 8:14 AM | Permalink | Comments (0)

Admin

• Oracle TAR 4947672.993: Respond to falacious assumptions on the part of the Oracle Support team
• Customize slurpd_status.pl for nagios monitoring
• Set up another account for SAMBA access on giraffe as part of the tele replacement project
• UPS's are delivered for Mann
• Work on LDAP update code to fix strangenesses with bad SSNs

Posted by at 8:43 AM | Permalink | Comments (0)

Admin

• Active Directory feed was broken yesterday and I missed it, fixed it today -- a recreation of a userid was performed that caused a problem
• FP Issue 17372: Local system Administrator is able to do the install of the Oracle client software... what a crock!

Posted by at 9:39 AM | Permalink | Comments (0)

Admin

• Recover the v9 windows oracle calendar client and palm calendar sync client for a customer who needs them while we figure out how to get the v10 versions working
• Experiment with forcing the removal of Calendar v9
• House Call -- unable to find problem and fix it. Need to open TAR and footprint about calendar client 10.1.2 install problem
  • TAR 4947672.993 opened with oracle.
  • They suggested a few things and then requested an iMeeting.
  • Working to schedule a time window when I can get access to the computer that has the problem to do the iMeeting

LDAP

• Work on building 2.2.30 for RHEL3

Posted by at 8:05 AM | Permalink | Comments (0)

Admin

• FP Issue=17330: Hung imapd
• rack1b full filesystem: moved two ugrads to other space

WebCT

• Install BigBrother monitoring
• Provide system admin support during installation of hotfix

Posted by at 8:33 AM | Permalink | Comments (0)

Admin

• Complete monetra removal from parrot

Planning

• Input to process enumeration of concerns

Posted by at 9:23 AM | Permalink | Comments (0)

Admin

• shutdown and disable monitoring of a webct backup machine
• lots of email exchange about Palm Zire syncing with Oracle Calendar Address Book information with a member of the A&S support staff
• Team Meeting

LDAP/Accounts

• Update scripts to create new mailRoutingAddress values
• Create script to convert existing entries to new values for mailRoutingAddress and add necessary values for mailLocalAddress
• Directory application updated to prevent display of posixaccount and inetMailRecipient attributes
• Ran script to do mass update of mailRoutingAddress and mailLocalAddress values

Posted by at 9:51 AM | Permalink | Comments (0)