Camelia, the Perl 6 bug

IRC log for #cqrs-perl6, 2012-09-18

| Channels | #cqrs-perl6 index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
19:36 masak joined #cqrs-perl6
19:36 masak o/
19:37 masak about to bring BooK in here.
19:37 masak he has some questions about building applications with events.
19:38 BooK joined #cqrs-perl6
19:38 masak \o
19:38 masak ok, so.
19:38 masak when you say "data", usually we say "aggregate".
19:38 BooK wait a sec
19:38 BooK dams will join
19:39 masak \o/
19:39 BooK what does cqrs mean, by the way?
19:39 BooK (while we wait for dams)
19:40 masak Command/Query Responsibility Segregation
19:40 masak the idea that reads (queries) and writes (commands) are different and merit being on different objects.
19:40 masak that's where it all starts, in a way.
19:41 masak but using events is pretty much orthogonal from that.
19:41 BooK ok
19:42 BooK (let's ignore dams, he'll join when he joins, that's what logs are for)
19:42 masak aye.
19:42 masak when I send a query, I always want data back. a report of some kind. when I send a command, I want to cause a change in the domain model.
19:43 masak the only thing I get back is direct feedback on whether I succeeded or not.
19:43 BooK ok
19:43 jnthn Not quite; often if you didn't suceed there's some kindication of *why*
19:43 jnthn er, indication
19:43 BooK so exception
19:43 masak oh, sure.
19:44 BooK with some information attached
19:44 masak though the main idea -- that the backflow from the commandhandler is very very narrow -- remains.
19:44 masak so commands and queries are the two ways a client can interact with a server. if you take pains to treat them differently, you're doing CQRS.
19:45 BooK isn't that always the case ?
19:45 BooK select vs insert in SQL, GET vs POST/PUT/DELETE in REST
19:45 masak most people mix them together on one interface, I think.
19:45 BooK so the events are basically a log of all the successful commands sent to the command handler
19:46 masak yes, but that's a backwards way of seeing it from my point of view :)
19:46 masak events are primary. commands are the way we generate them.
19:47 masak note that commands and events are not necessarily 1-to-1.
19:47 BooK how so?
19:47 BooK not all events generate a command?
19:47 masak they just aren't. they evolve independently as the domain model matures.
19:47 jnthn Well, largely events and commands have different design forces pulling on them
19:47 BooK (and I got it backwards in my previous question)
19:47 jnthn Events are the domain's API to the rest of the system
19:47 masak a command generates zero or more events.
19:48 jnthn Commands tend to be driven by the tasks performed from the UI
19:48 masak jnthn is right, though I haven't seen in practice what that means.
19:48 jnthn I've found that even when I've started out with them very similar, they diverge.
19:48 masak aye.
19:48 masak I think I have too little experience with command design still. need to do more UI work :)
19:49 jnthn masak: Well, practical example is when the aggregate is in a good place to include stuff it's figured out in an event that wasn't originally in the command. Or that it can build up from the history of events on that aggregate.
19:49 masak oh!
19:49 masak yeah, I got that in the adventure game.
19:49 masak dang, I've realized that before :P
19:49 BooK so the adventure game would be a good example to look at?
19:49 masak I had the command saying walk<north> and the event saying enter<chamber>.
19:49 jnthn Well, you get to the point where you do these things without conciously realizing it :)
19:50 masak BooK: the adventure game uses events internally, but because it's client-only it doesn't do the whole thing with commands and command handlers.
19:50 BooK masak: context + command = event, it seems
19:50 masak well, with "context", we're zeroing in on aggregates, I think.
19:51 BooK state
19:51 masak right.
19:51 masak aggregates make a model of the past in order to help command handlers validate commands.
19:51 BooK past or present ?
19:51 masak past.
19:51 BooK state would be the present
19:52 masak well, the present in this sense is just a narrow special case of the past :)
19:52 BooK hehe
19:52 BooK mmm, events take the UI out of the equation
19:52 masak if we store all the events for an aggregate, we have its whole history, and can do validation not just on the current state, but on prior states, too.
19:52 dams joined #cqrs-perl6
19:52 masak yeah, events are completely UI-agnostic.
19:52 dams heya
19:52 masak o/
19:53 dams am I very late ?
19:53 masak might be a good idea to backlog.
19:53 BooK one uses the UI to interact with the commands, which generate events, and then you can disconnect from the user, since you have all the effects of his actions on the world/model
19:53 dams can't, I just joined :/
19:53 masak in the common digram of this, events sit at the top (going from the write side to the read side), and the UI is at the bottom.
19:53 BooK dams: sending you the backlog
19:53 dams thanks
19:53 masak it's also online.
19:54 masak http://irclog.perlgeek.de/cqrs-perl6/2012-09-18
19:54 masak moritz++
19:56 BooK \o/
19:56 * BooK sends some lovin' to the bots
19:58 masak this is the diagram I meant: http://cqrs.files.wordpress.com/20​10/11/image14.png?w=550&amp;h=416
19:59 * masak waits for BooK|dams to ask more questions :)
19:59 * dams just finished backlogging
19:59 BooK I saw that diagram on your slides too
19:59 masak part of this post, which might be of interest: https://cqrs.wordpress.com/d​ocuments/cqrs-introduction/
19:59 masak as might the rest of that site, I guess.
20:00 dams question 1, naively : what are command handlers ? I sense there is a precise description of it, instead of my blurry (mis)representation of it
20:00 jnthn Coomand handler gets a command in, and decides what events / exception are gonna emerge.
20:01 dams oh ok, simple enough
20:01 masak command handlers are adapters. they turn a command into events-or-exception.
20:01 BooK jnthn: so they could be methods on an object?
20:01 jnthn BooK: They can be that, or you can do something more messaging-style.
20:02 jnthn (e.g. a command is a DTO of some kind)
20:02 jnthn Either is workable, it really depends on the scenario
20:02 masak the thing that is called a command handler is usually a class. there's a 1-to-1 relation between aggregates and command handlers.
20:03 jnthn yeah, that's clearer
20:03 BooK a class, not an object (instance) ?
20:03 masak one command handler declares which types of commands it handles. in C#, this is done using generics.
20:03 dams jnthn: hm when you say "command is a DTO of some kind", that sounds weird, as DTO are only for data transfer
20:03 dams how could they carry some "action" concept ( a command) ?
20:03 masak commands are DTOs.
20:03 jnthn dams: The action is all in the naming.
20:03 masak dams: this is actually an old Smalltalk pattern.
20:03 jnthn which for commands is in the imperative.
20:04 masak dams: for any given method call, you can "freeze" the call and serialize it to just name + arguments.
20:04 dams ah ok, got it
20:04 masak dams: put in another way, method calls are verbs, and commands are nouns representing those verbs.
20:04 BooK then it shouldn't really matter if you keep the events or the commands
20:05 masak BooK: events are primary.
20:05 dams I see now, that allows to use DTO implementation to actually carry commands, no need to specifically reimplement a special pipe
20:05 BooK but differents commands could generate the same events
20:05 masak oh, sure.
20:05 masak it makes sense to log the commands, too. they carry information.
20:05 masak especially if they fail.
20:05 jnthn Right, but they're not authoritative
20:05 BooK so the "what really happened" is the list of commands
20:06 masak no! :)
20:06 jnthn The events are the information authority
20:06 jnthn Commands are request, so logging them is like making a request log to a web server or so.
20:06 masak the events are the primary notion of "history" in the system.
20:06 BooK ok "what was requested" / "what really happened"
20:06 jnthn It doesn't tell you what was decided, only what was requested.
20:06 masak BooK: right.
20:06 dams the fact that a given command can end up generating events or exception is bound to the state at the given time
20:06 BooK so you need both if you want the full history
20:06 dams so replaying them later may end up with different events. is that correct ?
20:07 masak once an event is generated, you're way past all "immunodefense" barriers of the system.
20:07 jnthn You shouldn't replay commands.
20:07 BooK right
20:07 BooK you replay events
20:07 jnthn Events are the replayable things.
20:07 masak dams: right.
20:07 dams yeah
20:07 masak *because* commands depend on the state.
20:07 jnthn Because they're on the "decided" side of the fence.
20:07 masak a fence which is very noticeable in the code, by the way.
20:08 jnthn aye
20:08 BooK so, as I said earlier commands + state => events
20:08 jnthn afk for a bit, back later
20:08 dams and hm, how do you handle the database then?
20:08 BooK and the accumulation of events make the state change
20:09 masak dams: you can keep the database as 3NF or whatever, and "project" events down onto state. or you can store the events directly, and build state from that.
20:10 masak BooK: indeed. current state is a left fold of the history of events.
20:10 BooK 3NF >
20:10 BooK ?
20:10 dams I assumed state = the database at a given time. But it's not very handy
20:10 BooK right, events make the time tick
20:10 dams is there any formal representation of the state ?
20:10 masak BooK: third normal form. something that schools beat into us.
20:11 dams googling 3NF
20:11 BooK one thing dams and I were talking about, was to make a system distributed
20:11 BooK so basically reconcile state (sync / merge) from time to time
20:11 masak in brief 3NF is what database people tell us to make our SQL databases. then it's "well formed" in some sense. doesn't contain much duplication.
20:11 BooK ah normalization
20:12 masak BooK: I would be careful about having many owners of the same writeable data.
20:12 BooK and then we denormalize because fast is so much more important than correct :-)
20:12 masak though sync/merge is something that fascinates me too in relation to these ideas.
20:12 BooK I would think that most merges are trivial
20:12 masak depends on the domain.
20:13 BooK but of course, these are not the troublesome ones
20:13 masak well, one nice insight with CQRS is that normalization makes sense on the write side. denormalization makes sense on the read side.
20:13 BooK masak: indeed
20:13 masak old insight, but worth having in these times, as well.
20:13 BooK sometimes it makes sense to generate the data being read from the normalized / authoritative data set, so get it in a useable shape
20:14 dams but that can be done outside of the normalized / authoritative database
20:14 masak indeed.
20:14 BooK indeed
20:14 dams to avoid poluting it
20:14 masak that's what read sides are about.
20:15 dams so it's actually a non issue
20:15 masak they're "projections" of the write side and/or of the history of events.
20:15 dams question : I would naively keep a counter somewhere in the database, to store "where we are at, regarding state"
20:15 BooK dams: that's how I thought about it, with regard to the distributed thingy
20:16 dams so that I can see quickly if a dataset is in a old state compared to an other instance of the same dataset (in an other DB for instance)
20:16 BooK and actually, git works like that: it's not a live system, it's a series of snapshots
20:16 dams yeah
20:16 BooK you only need to keep snapshots of your synchronisation points
20:16 dams but we don't want one SHA1 for the entire DB
20:16 BooK no
20:16 masak dams: re "where we are at", see second table of http://cqrs.wordpress.com/docu​ments/building-event-storage/
20:17 masak it's a nice thing that we only really need those two tables. one for all events, and one for all aggregates.
20:17 BooK dams: that's what I thought: you can't get away with a single sha1 of the current whatever, you needs sha1/unique identification of meaninful subsets
20:18 BooK (exactly what trees, blob are: meaningfull subsets)
20:19 dams masak: thanks, just read it
20:19 BooK ah, I recognize those from your talk too
20:19 masak yeah :)
20:20 dams indeed :)
20:20 dams ok hm so ...
20:20 dams implementing all this will take time and a lot of effort
20:20 dams is there already some cool libs / tools that implement all that for us ? :)
20:21 masak this is the point where Greg would say, "just copy/paste my 500 or so lines of code, and you're good to go"
20:21 BooK that doesn't sound hard :-)
20:21 masak "who needs frameworks?"
20:22 BooK the interesting bit is that events are sequential
20:22 BooK so it looks hard to map with a distributed model
20:22 BooK s/with/to/
20:22 masak well, events for a given aggregate are sequential.
20:22 BooK are aggregates independant from each other ?
20:23 masak between aggregates, relativistic effects apply, and before/after ordering may not always be meaningful.
20:23 masak aggregates are *completely* independent from each other.
20:23 dams oh
20:23 BooK mmm
20:23 dams no relations between them ?
20:23 masak you may not update one on the basis of another being updated.
20:23 dams it would be wrong to have a talk aggregate that links to a speaker ?
20:23 masak all such interactions must go the way via events.
20:24 BooK like speaker has_a talk, that means they are both part of the same aggregate ?
20:24 masak BooK: well, that leads deep into aggregate modeling, which is non-trivial and interesting.
20:24 BooK I guess the important event is the one that says "this talk will now be given by that OTHER person"
20:24 masak but yes, an aggregate contains enough information to sustain it for a lifetime. it never goes outside of itself looking for more information.
20:24 BooK ah
20:25 dams masak: what's the proper way to handle relationships then ?
20:25 BooK masak: but then most application end up having a single aggregate?
20:25 dams usual RDBM is ok ?
20:25 masak BooK: no, that's the bastardized form of aggregate design :)
20:25 BooK in the conference model we were talking about, how would that work>
20:26 BooK a speaker is an aggregate and a talk is another?
20:26 masak depends on your use cases and your other requirements.
20:26 BooK but if aggregates were objects, you'd call them that :-)
20:26 masak but yes, separate aggregate types sounds more sane there.
20:26 masak many of the actual "compositions"/mashups of information can be done in the read models.
20:27 masak so if you're looking for where the JOINs are made, it's there, in the projections down to a read model.
20:27 BooK because $person is going to give $talk doesn't intrinsically change any of them
20:27 dams ok. But it's still ok for a talk to have an id pointing to a speaker ?
20:28 BooK seems like the relation is an aggregate
20:28 dams ah ok
20:28 masak BooK: well, maybe there's a third aggregate type which references both of them. or maybe $talk knows the $person's name, and that's enough.
20:29 masak dams: yeah, but the "pointer" here is just a string UUID or something.
20:29 dams I tend to dislike these data models where the relationships are seperated from the data
20:29 masak dams: enough for read views to connect the dots.
20:29 dams ok, so it's OK if it's a proper UUID, but it must not be a database id. I see
20:29 BooK dams: from what I understand, there would be two databases
20:29 masak well, I needed to say first that aggregates are *completely* independent, and then gradually introduce "symbolic references" through UUIDs :)
20:30 BooK the write one, and the read one
20:30 masak right.
20:30 masak the write database contains all the aggregates in some form.
20:30 masak the read database(s) hold "reports", ready-made presentations/mashups of those aggregates.
20:30 dams but the read one can be completely regenerated from the write one, right ?
20:30 masak aye.
20:30 dams ok cool
20:31 masak it's secondary information.
20:31 masak you can drop any read side and re-project it from the history of events.
20:31 BooK in a database model, the aggregates would be each normalized table and each link table
20:31 BooK rigth?
20:31 masak aggregate != table
20:31 dams it's funny to see how big web app eventually come to this kind of design
20:32 BooK I have a hard time trying to think of an aggregate that does not map to a table
20:32 dams people building a cache to pregenerate read data, then add more intelligence to it, ending up having a real read DB asside
20:32 masak BooK: I started my understanding of aggregates with what they did: they prevent outside access, and favor copying to linking.
20:32 dams BooK: the list of non english speaking speakers that give a talk in an english speaking room of a conference
20:32 masak BooK: but now I think of them as jnthn thinks of them: they are there to help the command handler validate the command, and do this by building a model of the past.
20:33 masak BooK: of course you can map anything to a table, if you want. but I don't think that's a helpful way to think of it.
20:33 masak BooK: aggregates are defined not by how they look in the DB, but by how they look in memory.
20:35 BooK whats the relationship betweem aggregates and objects?
20:35 masak I tend to see an aggregate as a self-contained graph of objects.
20:36 masak aggregates have a *root*, which is a kind of facade/representative for the whole object graph.
20:36 masak the root is the only thing that the outside world is meant to call methods on.
20:37 BooK what about the 1-to-1 mapping of aggregates and command handlers?
20:38 dams does the word "aggregate" comes from the fact that it's an aggregation of objects?
20:38 masak BooK: well, we have a slight group/individual distinction problem here, I realize.
20:38 masak dams: yeah, like a bunch of grapes.
20:38 masak BooK: one command handler (class) may serve one aggregate (class).
20:38 dams I see. I had a wompletely wrong idea of what they were, then
20:38 BooK if command handlers are classes, and objects are the binding of a command hanlder to a state, what are the aggregates?
20:38 BooK aha! aggregates are classes too
20:39 masak yeah. in code, an aggregate root is a class.
20:39 masak and it encapsulates the rest of the object graph as private attributes.
20:39 BooK speaking of class, what about giving a long lecture on the topic at osdc.fr?
20:39 dams at some point I'd like to see some implementation :/ is there a simple example you'd recommend ?
20:40 BooK longer than your yapc eu 2011 talk :-)
20:40 masak dams: I might dig up Greg's example. it's in C#.
20:40 masak oh!
20:41 dams masak: if the aggregate root is a class, I guess an instance of it would be the actual "grapes of objects", right ?
20:41 masak we even have one written in Perl. :)
20:41 dams awesome
20:41 masak dams: right.
20:41 masak https://github.com/jnthn/hex
20:41 masak pay special attention to
20:41 masak an aggregate root: https://github.com/jnthn/hex/blob/m​aster/lib/Hex/AggregateRoot/Game.pm
20:42 masak a command handler: https://github.com/jnthn/hex/blob/ma​ster/lib/Hex/CommandHandlers/Game.pm
20:42 masak awesome BDD-style tests: https://github.com/jnthn/he​x/blob/master/t/gameplay.t
20:42 dams ok, and the aggregate root constructor is supposed to receive the minimal information to build the "grapes". And the command handlers is in charge of providing the minimal information ?
20:42 masak also note, this project is 1.5 years old. jnthn and I have some more specified knowledge about the factoring of these things now, which has not gone into this example.
20:43 masak dams: yeah. usually a command causes the aggregate to be created.
20:43 dams yeah, ok I understand
20:44 dams ok so actually there is no strong relation to an aggregate and how the data objects are stored in the DB
20:44 dams or somewhere else for what it's worth
20:45 masak not necessarily, no.
20:46 BooK mmm, apply versus handle
20:46 masak but we don't have much of a need for something like an ORM, because if we wanted we could just stick a serialized version of the aggregate into the DB.
20:46 dams okay. before I forget, thanks a lot for the time spent here, much apreciated
20:46 masak BooK: command handlers handle commands. aggregates apply events.
20:46 masak dams: no problem, this is very helpful to me/us, too.
20:46 BooK yes, thanks a lot
20:46 masak especially the fact that it gets logged and we can go back to it later and see what explanations helped.
20:47 masak we're building courses around this and need nice ways to explain all this stuff.
20:47 BooK dams and I are thinking about starting a new project together, and we have lots of ideas
20:47 dams aha yes I'm happy you're saying we don't need an ORM, I was too shy to emit the idea :)
20:47 BooK masak: wanna give a course in Paris in October?
20:47 masak an ORM is problematic because it also wants to be primary, but the events are primary.
20:47 dams aye
20:47 BooK it can be primary at handling the secondary source
20:47 masak BooK: insufficient data. my October schedule is partly booked ;)
20:48 BooK 12-13 October
20:48 masak sorry; am away then.
20:48 masak would've loved to come talk.
20:48 BooK s/secondary source/secondary data/ # i.e. the read data
20:48 dams when are you available ?
20:48 masak subsequent weekends, basically.
20:48 BooK what about jnthn ?
20:49 dams hmmm
20:49 dams masak: there is the Quack & Hack that we organize this year in Paris
20:49 dams 7->9 december
20:49 BooK that could be part of the "quack" :-)
20:49 dams yeah
20:49 BooK I want it recorded!
20:49 masak 7-9 December sounds more doable.
20:50 BooK (I can't attend quack and hack)
20:50 dams \o/
20:50 masak and I'm intrigued by the chance to come give a long CQRS/ES talk.
20:50 masak we'll think of something :)
20:50 dams BooK: sure you can. It's just a wife-convincing action away !
20:50 BooK dams: my jedi mind tricks do not work on her
20:50 dams this quack and hack will be good, there will be a dancer2 hackathon too.
20:51 * BooK so want to be there
20:51 dams speaking of which, I'm afraid I have to leave, I need to finish coding some stuff :/
20:51 dams and I could read more stuff about CQRS.
20:52 dams masak: however, before I leave, I'm troubled by the fact that it breaks a lot the current design we apply
20:52 masak I'll stay on this channel.
20:52 masak just ping me if you have more questions later.
20:52 masak breakages of current designs are problematic, yes.
20:52 masak do not attempt to apply this pattern to everything :)
20:53 masak there are some things it does well. likewise, there are some things a pure CRUD design does well.
20:53 dams thanks, I may ask more :)
20:54 dams yeah. Everytime I'm in a big design discussion, I mentally weight the new design versus an ugly CRUD quick and dirty then refactor design
20:54 dams most of the successful startups have succeded because they started quick and dirty...
20:54 dams anyway, thanks again, afk
20:55 masak o/
20:55 BooK actually, the AggregateRoot looks pretty much independant
20:56 BooK in ex
20:56 BooK hex
20:56 masak yes...
20:56 masak aggregate roots are meant to be independent.
20:57 BooK I meant it could be in a CQRS:: namespace
20:57 masak I do not understand.
20:58 BooK Hex::AggregateRoot::Game is domain specific
20:58 BooK but Hex::AggregateRoot could be CQRS::AggregateRoot, as it contains no domain-specific stuff
20:58 masak ah.
20:58 masak yes.
20:58 masak sure.
20:59 masak though aggregate roots are only indirectly related to CQRS.
20:59 BooK same for the EventAggregator
21:00 BooK I meant they provide generic stuff to be used in any application
21:01 BooK I would add Event to the list, but Event is empty... :-)
21:04 BooK don't you think you could wrap those in a CPAN distribution? with some pointers to more documentation?
21:05 BooK I note that the Hex game lacks a UI :-)
21:06 BooK funny thing, is that for the project dams and I are working on, I really wanted to start without a UI at all, as I think it is secondary (and there are more than one)
21:10 BooK the hex example is very enlighting
21:10 masak maybe some day I will push a CPAN distribution, yes.
21:10 BooK reading the test suite, especially
21:10 masak again, note that our understanding of these things has evolved a little since then.
21:10 masak our factoring nowadays is slightly simpler.
21:11 BooK the given, when, then I saw in the tests was also in your slides
21:11 BooK simpler, how?
21:11 BooK that looks pretty simple already
21:11 jnthn Wsa gonna say. We learend things since Hex :)
21:11 jnthn BooK: Somebody came up with the idea that the command handler returns the events to apply. It turns out it's a big cleanup :)
21:12 masak yeah, it turns all FP-y and nice.
21:12 jnthn Well, the insight in some of this is that you're doing FP more than you're doing OO
21:13 jnthn Applying events to an aggregate is just a fold.
21:13 BooK I know fold is FP, but I don't remember exactly what
21:13 jnthn Well, like [+] 1,2,3 in Perl 6 :)
21:14 BooK reduce ?
21:14 jnthn It turns a list of things into a single value somehow.
21:14 BooK ok
21:14 BooK ah yeah, in hex the command handler was actually changing the state
21:16 masak right.
21:16 masak that's how we used to factor it back then. :)
21:17 BooK interesting: the Command/ResignGame has no player paramater
21:17 masak hm.
21:18 masak it could well have that.
21:18 BooK but the corresponding event has it
21:18 masak especially since in a real-time game, either player can resign at any time.
21:18 BooK at least in the test suite
21:18 masak sleepy-time here. 'night
21:18 BooK thanks very much for your time
21:19 BooK will need to sleep on it too :-)
21:19 BooK you guys really must talk about this more
21:19 jnthn :)
21:19 BooK jnthn: did you see dams and I trying to get masak to talk about this at osdc.fr
21:19 BooK what about you?
21:20 BooK also busy on october 12-13?
21:20 * jnthn actually is spending time with a $dayjob client that he's mentoring on these patterns tomorrow :)
21:20 jnthn BooK: I...think those are he dates of (or overlap with) the Nordic Perl workshop, which I already promised to speak at.
21:20 jnthn *the
21:20 BooK ok
21:21 jnthn Anyway, yes, I'm generally happy to speak on these topics. Teaching it is kinda part of my job now :)
21:21 jnthn But I'm happy to share what I figured out so far at confs and so on of course :)
21:35 BooK what about providing advice to people writing a large project on those ideas?
21:36 jnthn Can do, but if it's that large you probably want to have the bounded context discussion before this :)
21:37 jnthn Well, I should get some rest...
21:42 BooK good night
21:42 BooK and thank you again

| Channels | #cqrs-perl6 index | Today | | Search | Google Search | Plain-Text | summary