Perl 6 - the future is here, just unevenly distributed

IRC log for #openam, 2014-11-19

| Channels | #openam index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
05:22 Hunger- joined #openam
05:50 ramteid joined #openam
07:27 aldaris joined #openam
11:33 kala hello. Could someone, maybe aldaris, clarify the CTS stuff. http://openam.forgerock.org/openam-documentation/openam-doc-source/doc/webhelp/install-guide/cts-general-recommend.html says that "The CTS requires that it talks to a single server to reduce the risk of replication errors.", even when the CTS configuration takes the connection string, which can specify multiple servers
11:34 aldaris sounds clear to me
11:35 kala so, even when I specify two ldap servers for the external CTS, the OpenAM will still connect to just one?
11:36 kala and it expects primary-secondary replication, but multi-master replication from the LDAP servers?
11:36 kala s/but/not/
11:37 aldaris s/primary-secondary/master-slave to correct the terminology
11:37 aldaris you can't specify more than one CTS server on the CTS tab
11:37 aldaris the server field is singlevalued
11:37 aldaris on 11 at least
11:38 aldaris you can only have more than one server defined for CTS if you use the default token store mode, which will mean the configuration stores should be used for CTS as well (only a different suffix)
11:38 kala yes, I'm just reading the docs, I haven't looked at the console itself
11:39 kala hmm
11:41 kala well. we try to run the OpenAM in "shared-nothing", "any single component can break, but service still works" architecture, with LDAPs being multi-master replicated. We lately discovered that this means that we will need some kind of locking, because multiple requests could come in at the same time and get processed on separate openam instances and we end up inconsitent user data
11:42 kala with LDAP "assured-replication" this might be bit better, but it might still be a good idea to have a locking feature. And jjpp was wondering if we could implement something on top of the CTS
11:43 jjpp (and conflicts are there partly because we have hacked profile management into openam because openidm was not invented when we first set it all up)
11:43 jjpp otoh, we still have not studied openidm enough to say that it will not suite for our usecase anyway :)
11:44 kala right, but it is still nagging you nevertheless
11:44 kala aldaris: so ... there is currently no "general, global, syncronous, ..." locking service in OpenAM?
11:45 aldaris no
11:45 aldaris and we are not particularly keen to have one
11:45 kala aldaris: and it seems that CTS is also not aimed to be syncronous, global, fault-tolerant, distributed database
11:45 jjpp kala: and using cts is just one possibility. using any read-safe replicated ldap tree would probably be as good
11:46 kala jjpp: yes, I just figured that maybe we would use some OpenAM classes. CoreTokenServiceAdapterSomething
11:46 kala aldaris: not particularly keen to have one? Because you reckon this is not required or openam is not the correct place for that?
11:46 jjpp just using ldap sdk would probably be easier
11:48 aldaris for the user management as well! :)
11:49 jjpp possibly. although in that departement I don't really see what it would give us more. There is not that much of idrepo api to work around in one hand
11:49 jjpp and there is nothing we could not do so far that direct ldap would let us to do
11:50 jjpp otoh, to build locks, one could use this conditional extension that allows to do test-and-set. whatever it was called. :)
11:51 jjpp and as (at least in openam11) no api that wraps ldap uses it in openam, one has to go directly. :)
11:52 aldaris did you mean etag control?
11:53 jjpp also, we have multiple ways to authenticate per profile and ways to manage those multiple identities. and most of the operations related to that are closely related to authenticating with those identities and then building/changing sessions on what user decides to do.
11:53 jjpp nope
11:54 jjpp there was something that allows you to add filters to requests
11:54 kala "Assertion Request Control" - 1.3.6.1.1.12
11:55 jjpp yup
11:55 jjpp but, the team needs me to go to the lunch. afk for a while.
12:27 kala hmm, we are lucky. Mark Craig has just documented this Assertion Control in a blog post http://marginnotes2.wordpress.com/2014/10/13/
14:10 insaniOpenAM joined #openam
14:11 insaniOpenAM hi all. Is there a way to retrieve user data from more than one different backends/resources ?
14:11 MegaMatt OpenIDM
14:12 insaniOpenAM I mean: for the same realm and agent. I have some data from ldap and the remaining data is in another source. I'd like to retrieve both so that I could authorize that user to my app.
14:13 insaniOpenAM I've read a bit about openim but I'm not sure I'd need it for what I'm talking about.
14:14 insaniOpenAM I need the same kind of authorization I already do with openam - but user data now comes from more than one source (i.e.:ldap + oracle database)
14:14 MegaMatt pretty sure you could just query IDM and it would give a reconciled answer
14:15 insaniOpenAM username/password comes from ldap. groups information comes from oracle database.
14:15 insaniOpenAM then, I'd mix both and authorize or deny that user.
14:15 jjpp can't you just configure multiple id repos for realm for that?
14:16 MegaMatt Yeah, you can configure both repos and it will query both
14:16 insaniOpenAM jjpp: not sure. never done that before. is that something simple ?
14:16 insaniOpenAM by "simple", I mean "as simple as what we already do with openam" :_)
14:16 jjpp the real problem will be probably i the jdbc id repo works well enough
14:17 MegaMatt Yeah, it’s simple.
14:17 insaniOpenAM MegaMatt jjpp: any docs on how to configure repos ?
14:17 MegaMatt but just remember everything you do will go against both datasources
14:17 insaniOpenAM MegaMatt: right. that's a requirement the team is aware of.
14:17 jjpp insaniOpenAM: the usual openam docs, I believe
14:18 insaniOpenAM Chapter 18. Customizing Identity Data Storage ?
14:18 jjpp http://docs.forgerock.org/en/openam/11.0.0/admin-guide/#realm-data-store
14:18 insaniOpenAM hmm. nice.
14:19 insaniOpenAM I'll read it. thanks, guys.
17:40 insaniOpenAM people: I'm trying to change some stuff in openam's default UI
17:40 insaniOpenAM I see the documentation on how to find the pages for tomcat
17:41 insaniOpenAM any hints on where they should be for weblogic ?
17:42 insaniOpenAM ops. I think I got it.
18:17 aldaris joined #openam
18:25 pcypher joined #openam
18:49 aldaris joined #openam
18:55 pcypher joined #openam
19:36 aldaris joined #openam
20:10 aldaris joined #openam
20:22 sbrooks9 joined #openam
20:37 sbrooks9 I'm trying to install openam 11 and keep getting an error when configuring directory server. The error I get is: configurator.embsetupopendsfailed. I get this error with both the default configurator and when I configure it manually. Any idea what the issue would be?
20:37 MegaMatt What’s your permgen setting and heap setting for the container?
20:46 sbrooks9 xmx = 1024m and maxpermsize = 256m, because I ran the command catalina_opts ect?
20:48 MegaMatt Are you connecting to the url of the box it’s on?
20:48 MegaMatt So that it can pick the right portS?
20:48 MegaMatt or are you connecting to an IP or some other url ?
20:48 sbrooks9 some other example url
20:49 MegaMatt So does that resolve on the machine?
20:50 MegaMatt My guess is that the URL doesn’t resolve on the openam machine, so it’s not picking good port #s or something like that
20:50 sbrooks9 I added the url to the /etc/hosts file as directed from the getting started pdf
20:51 MegaMatt Ok so in /etc/hosts you have say openam.example.com pointing to localhost or something similar, right?
20:51 sbrooks9 correct
20:51 MegaMatt And you’re on that localhost, brining up a browser and connecting to openam.example.com/openam ?
20:51 sbrooks9 correct, I can see the configuration page
20:52 MegaMatt ok, then when you go through the manual configuration, does it select good ports for opendj?
20:54 sbrooks9 it doesn't throw an error and say it's wrong
20:54 MegaMatt but is it selecting ports like -1 or something below 1024 and you’re not root?
20:55 MegaMatt I bet it’s your JVM options though
20:55 MegaMatt that happened to rghose
20:56 MegaMatt they thought they had set permgen, but the settings weren’t getting picked up by TC
20:56 MegaMatt he had to restart TC to get it to pick up the JVM options
20:56 MegaMatt see: http://irclog.perlgeek.de/openam/2014-05-07
20:57 MegaMatt Otherwise, check your install.log file for more details
20:57 sbrooks9 yea, the ports are all above 5000
20:58 MegaMatt ok, then I’d really check ps and/or jinfo to make sure the jvm settings were picked up
20:58 MegaMatt and check the install.log file for more details
20:58 sbrooks9 okay, so it
20:58 sbrooks9 * opps, so it's definitely something with a setting before openam?
20:58 MegaMatt usually it’s the permgen setting that causes problems on deployment
20:59 MegaMatt but it can be something else, gotta check the install.log
20:59 MegaMatt Most likely you didn’t really set the maxpermsize yet
20:59 MegaMatt jinfo can tell you
20:59 MegaMatt or sometimes you can see it in ps
21:01 sbrooks9 well the install log tells me there is an error confguring the directory server and failed 5
21:01 MegaMatt and the first suggestion yields what result? ;)
21:05 sbrooks9 jinfo can't attach to the tomcat pid
21:05 MegaMatt how about a simple ps?
21:06 sbrooks9 I get 2 pids, none of which equal my tomcat pid
21:06 MegaMatt ?? What OS is this btw?
21:07 sbrooks9 ubuntu
21:07 sbrooks9 14.04
21:07 MegaMatt my ps looks like this
21:07 MegaMatt r$ ps -ef |grep java
21:07 MegaMatt 501 33482     1   0  4:05PM ttys000    0:41.49 /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/bin/java -Djava.util.logging.config.file=/usr/local/Cellar/tomcat/7.0.47/libexec/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -server -Xms3g -Xmx3g -XX:MaxPermSize=628M -Djava.endorsed.dirs=/usr/local/Cellar/tomcat/7.0.47/libexec/endorsed -classpath
21:07 MegaMatt /usr/local/Cellar/tomcat/7.0.47/libexec/bin/bootstrap.jar:/usr/local/Cellar/tomcat/7.0.47/libexec/bin/tomcat-juli.jar -Dcatalina.base=/usr/local/Cellar/tomcat/7.0.47/libexec -Dcatalina.home=/usr/local/Cellar/tomcat/7.0.47/libexec -Djava.io.tmpdir=/usr/local/Cellar/tomcat/7.0.47/libexec/temp org.apache.catalina.startup.Bootstrap start
21:07 MegaMatt So I can see my permgen and xmx
21:08 MegaMatt jinfo probably can’t attach because ubuntu has some funky control thing on by default
21:09 sbrooks9 I apologize, I executed the command differently. my xmx says -Xmx128m
21:09 MegaMatt # Ubuntu 10.10 and newer has a new default security policy that affects  strace, jmap, jinfo, and other Serviceability commands. This policy prevents a process from attaching to another process owned by the same UID if the target process is not a descendant of the  attaching process. The following will change the kernel's yama/ptrace_scope variable temporarily, i.e., until the next reboot:
21:09 MegaMatt $ echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope
21:09 MegaMatt
21:09 MegaMatt For more info see: http://bugs.java.com/bugdatabase/view_bug.do?bug_id=7050524
21:09 MegaMatt so your xmx is low, what about your MaxPermSize?
21:10 MegaMatt -Xmx of 128m is pretty darn small.. should probably be 1024m like you thought it was at the start, and maxpermsize should be 256m like you thought ;)
21:11 sbrooks9 interesting.. neither are set. Any idea how to do that without using that cataline_opts cammand that didn't work before?
21:11 MegaMatt I make a setenv.sh in my bin directory for tc
21:12 MegaMatt inside the setenv.sh I put: export JAVA_OPTS="$JAVA_OPTS -server -Xms3g -Xmx3g -XX:MaxPermSize=628M"
21:13 MegaMatt but you can set whatever values you want in there
21:13 MegaMatt Mine are a bit overkill
21:13 MegaMatt I think I was messing with something and set it that way for a test
21:15 sbrooks9 okay, I did the same. now it should configure?
21:15 aldaris joined #openam
21:15 MegaMatt yeah, when you restart tc it should pick up those options
21:15 MegaMatt just double check it with ps again ;)
21:17 sbrooks9 was the command suppose to be -xm(with an s here)3g? My tomcat isn't booting back up now
21:17 MegaMatt No you don’t have to go to 3g
21:17 MegaMatt you can do -Xms1g -Xmx1g -XX:MaxPermSize=256m
21:18 MegaMatt especially if this is just an evaluation type server
21:18 MegaMatt Like I said, I just have my values cranked up
21:18 sbrooks9 yea, the question was more do I use xmx both times or did you mean xms the first time
21:18 MegaMatt Xms and Xmx
21:19 MegaMatt You set them equal
21:19 MegaMatt Xms is the starting heap, Xmx is the max heap
21:19 MegaMatt You set them equal so it doesn’t have to grow the heap over time
21:19 MegaMatt and it allocates the entire heap at startup
21:19 MegaMatt The key one is the MaxPermSize
21:20 MegaMatt Because I think server picks 1G by default anyhow for a heap...
21:20 sbrooks9 hm, tomcat still won't start back up, even with lower values
21:21 MegaMatt what’s the error in catalina.out
21:21 MegaMatt typo in the options maybe?
21:23 sbrooks9 insuffient memory for java runtime
21:23 MegaMatt ouch
21:23 MegaMatt how much ram is on this machine?
21:24 MegaMatt 1g is not a lot .. heh
21:24 sbrooks9 1g.. I'm an intern and this is the system they set me up with to configure this on haha
21:24 MegaMatt that’s pretty bad
21:25 MegaMatt 1g of ram is quite pathetic
21:25 sbrooks9 so more ram is a must?
21:25 MegaMatt You can TRY lowering the Xmx and keeping the permgen high
21:25 MegaMatt but it’s going to run like crapola
21:26 MegaMatt Try setting like -Xms500m -Xmx500m -XX:MaxPermSize=100m
21:26 sbrooks9 as long as I can get it to configure, I'll be happy lol
21:26 MegaMatt Do you even have 600mb of free ram?
21:26 MegaMatt heh
21:27 MegaMatt The requirements state 1g of ram… just for the java heap
21:28 MegaMatt A machine with 1g total ram. .. well, it’s going to struggle ;)
21:29 sbrooks9 haha it started with the lower specs
21:29 MegaMatt yeah, but it might not deploy
21:29 MegaMatt well, configure that is
21:29 MegaMatt with the embedded dj
21:29 MegaMatt You might be able to get away with an external config store
21:29 sbrooks9 yea.. dj is where it was failing in the configuration
21:29 MegaMatt but really, you’re cutting things really tight
21:30 MegaMatt see: http://docs.forgerock.org/en/openam/11.0.0/install-guide/#prepare-java-sun
21:30 sbrooks9 okay, we'll I'll talk to my boss about more ram. However, i did a ps on java again and it doesn't look like the file I made changed the xmx from 128 or the maxpermsize
21:30 sbrooks9 well*
21:31 MegaMatt so it must be picking it up from somewhere else .. or the file permissions aren’t right
21:31 sbrooks9 wait nvm, it worked. :) thank you. once the machine has more ram, it should be good from here?
21:31 MegaMatt but it did sound like it was
21:31 MegaMatt since you got the error before ;)
21:31 MegaMatt Yeah, I think you should be fine if you can set Xmx=1g and PermSize=256m
21:32 MegaMatt I’ve deployed a lot of times using those settings...
21:32 MegaMatt and failed a lot of times when PermSize wasn’t set ;)
21:32 sbrooks9 okay, fantastic. Thank you very much! You've been really helpful. I would have never narrowed it down to a ram issue haha
21:32 MegaMatt np, gl
21:41 sbrooks9 I am now the proud owner of 4gigs of ram and a configurated openam (no errors!). Thanks again!
21:42 MegaMatt ;) yw
21:54 aldaris joined #openam
22:00 pcypher_ joined #openam
22:42 pcypher joined #openam
23:27 sbrooks9 after the web policy agent is installed and configured, you shouldn't be able to see any tomcat pages, correct?

| Channels | #openam index | Today | | Search | Google Search | Plain-Text | summary