Perl 6 - the future is here, just unevenly distributed

IRC log for #fuel, 2016-07-28

| Channels | #fuel index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:06 Sesso_ joined #fuel
00:10 elo joined #fuel
00:19 kaliya joined #fuel
00:43 code-R joined #fuel
00:50 xarses joined #fuel
01:07 code-R_ joined #fuel
01:26 johnavp19892 joined #fuel
01:46 gbraad joined #fuel
03:26 Liuqing joined #fuel
03:51 kaliya joined #fuel
04:00 fritchie joined #fuel
04:36 javeriak joined #fuel
04:48 javeriak joined #fuel
05:04 fritchie joined #fuel
05:25 javeriak joined #fuel
05:25 javeriak joined #fuel
05:37 javeriak_ joined #fuel
05:37 dguryanov2 joined #fuel
06:00 fritchie joined #fuel
06:12 code-R joined #fuel
06:23 Miouge joined #fuel
06:25 Researcher- joined #fuel
06:29 code-R_ joined #fuel
06:43 code-R joined #fuel
06:57 javeriak joined #fuel
07:13 permalac joined #fuel
07:51 chopmann joined #fuel
08:00 fritchie joined #fuel
08:17 yassine joined #fuel
08:26 permalac joined #fuel
08:39 Liuqing_ joined #fuel
08:40 ikalnitsky ikutukov: asvechnikov would you mind guys to take a look? https://review.openstack.org/#/c/346908/
08:56 kelepirci joined #fuel
09:08 javeriak joined #fuel
09:30 ekosareva joined #fuel
10:00 fritchie joined #fuel
11:28 vkulanov joined #fuel
11:35 dguryanov2 fuel-web cores, could you, please, review https://review.openstack.org/#/c/332019/ ?
11:56 Liuqing joined #fuel
12:00 fritchie joined #fuel
12:24 gcmacmalloc joined #fuel
12:34 jackdastripper Good morning. Since there is a bug in 9.0 that won't allow you to reuse host names, how can you manually reset the nailgun database?
12:58 flerfb0rt joined #fuel
13:00 flerfb0rt joined #fuel
13:01 gcmacmalloc_ joined #fuel
13:04 flerfb0rt1 joined #fuel
13:12 mskalski jackdastripper: what do you mean by reusing hostname? you assigned hostname to node by fuel nodes --node 9 --hostname 'name' and you are not able to assign the same name to different node?
13:16 johnavp1989 joined #fuel
13:17 jackdastripper There is a bug in 9.0 that will cause the deployment to fail if you reuse a host name.
13:17 mwhahaha so we created a bug but couldn't reproduce it
13:17 mskalski jackdastripper: do you have a ling to this bug on launchpad?
13:19 mwhahaha https://bugs.launchpad.net/fuel/+bug/1604547
13:20 mskalski mwhahaha: thanks
13:20 mwhahaha jackdastripper: so if you have a specific way to reproduce it please update that bug and we can get it fixed
13:25 jackdastripper There is already a bug logged for it. I would have to search for it again as I have already reset my lab to keep moving forward. It also there is an issue with deployments failing if you select L3 HA.
13:28 johnavp1989 are there any octane expects hanging out in here today?
13:29 Kdecherf is there a way to add persistent iptables rules on deployed hosts?
13:29 mwhahaha not via fuel, but you can just manually (add it and & save)
13:30 mwhahaha we don't truncate the tables via fuel
13:41 code-R joined #fuel
13:42 code-R_ joined #fuel
13:52 Liuqing joined #fuel
13:54 ekosareva joined #fuel
14:00 fritchie joined #fuel
14:04 code-R joined #fuel
14:05 dteselkin joined #fuel
14:06 kelepirci hello all
14:06 kelepirci a simple question. we want to upgrade the kernel of compute and controller nodes
14:07 kelepirci will that couse any problem?
14:10 code-R_ joined #fuel
14:14 Liuqing joined #fuel
14:20 garrett_ joined #fuel
14:20 garrett_ hey everyone
14:21 Guest21752 for some reason my iso build is stopping and not generating the local_mirror/ubuntu-packages.changelog file
14:21 Guest21752 anyone know why that file would not be downloaded?
14:23 smatov joined #fuel
14:31 Guest21752 here is a link to my terminal output of why its failing
14:31 Guest21752 http://paste.openstack.org/show/543282/
14:33 Sketch rsync error 23 means some files could not be transferred
14:34 Sketch but i guess it's just operating on local files anyway, so it's just that one which is missing
14:35 Sketch i have no idea, but...is your internet access working on your fuel host?  IIRC it does download some files from the internet while generating images
14:35 Guest21752 right, so for some reason that file is not even generated
14:35 Guest21752 i have checked and it does not exist
14:36 Guest21752 yup ive had internet the whole time, one thing to mention though is i am using my own mirrors
14:36 Guest21752 is it possible that my ubuntu mirror is the wrong version or something?
14:37 jackdastripper Why does fuel force the DHCP pool range to be .3 to .254 on the interfaces of the master node? That violates network best practices. You need atleast three IPs for the switch VRRP. 1 virtual default GW and two for the vlan interfaces on the two switches.
14:38 mskalski jackdastripper: it is configurable
14:39 jackdastripper It always reverts back to .3 for me
14:39 jackdastripper in the fuelmenu
14:41 mskalski jackdastripper: during installation or after fuel-master is installed?
14:42 mwhahaha if it's reverting it's probably a bug. that being said generally we don't assume such configurations for the admin network as it's more of a management network and not a production one
14:42 mwhahaha but it should honor your configuration. it starts on .3 because the default for the master is .2
14:43 jackdastripper understand, but even management networks generally have redundant switches.
14:43 mwhahaha it can be accomplished in other ways, but yes
14:44 mwhahaha the point being it should be configurable and if it's not honoring the configuration please report a bug
14:44 mwhahaha https://wiki.openstack.org/wiki/Fuel/How_to_contribute#Test_and_report_bugs
14:44 jackdastripper fair enough. thanks
14:45 Guest21752 mskalski: mwhahaha either of you guys no why the ubuntu-packages.changelog file would not be generated?
14:47 mwhahaha nope
14:47 mwhahaha is this 9.0?
14:47 Guest21752 7.0
14:49 mwhahaha so that's supposed to get generated by the report-changelog.sh
14:49 mwhahaha https://github.com/openstack/fuel-main/blob/be7068e7d85a84d9c96f323130cca4cf818c5f1b/report-changelog.sh
14:49 mwhahaha which should get called here: https://github.com/openstack/fuel-main/blob/be7068e7d85a84d9c96f323130cca4cf818c5f1b/mirror/module.mk#L19-L22
14:50 mwhahaha you could try running that manually to generate it
14:52 xarses joined #fuel
14:52 Guest21752 so just try running the report-changelog script manually?
14:53 Guest21752 so if I run the script it seems to just hagn
14:53 Guest21752 hang
14:53 mwhahaha did you export LOCAL_MIRROR?
14:54 mwhahaha you could also edit the report-changelog.sh to add set +x so you can see where it's hanging
14:54 Guest21752 no, what should i export local_mirror as?
14:54 mwhahaha the local mirror folder path
14:55 mwhahaha so /home/garrett/Tasks/main2/fuel-main/local_mirror according to your paste
14:56 mskalski jackdastripper: fuel 9.0? I can't reproduce described behaviour change in dhcp range is saved
14:59 Guest21752 mwhahaha i have exported the variable, same behavior. Also i ran report-changelog with +x permission and it completed however there was no output
14:59 Guest21752 should i try a deep_clean with the variable exported? It looks like the script actually exports the local_mirror anyways?
15:00 mwhahaha no it should configure that correctly when ran
15:00 mwhahaha that's part of the module.ml
15:00 mskalski jackdastripper: do you use standard subnet 10.20.0.0/24? Or configure pxe on different interface, maybe there is some specific scenario when it does not work, when it back to .3 is there a message that verification failed or something like that?
15:00 akscram ikutukov, akasatkin, ikalnitsky, dpyzhov: Could you please review the next backport in stable/mitaka? https://review.openstack.org/#/c/348190/
15:00 mwhahaha no not permissions +x, edit file, add 'set +x' under #!/bin/bash
15:02 Guest21752 still no output
15:03 mwhahaha so it looks like http://paste.openstack.org/show/543292/
15:04 Guest21752 yes looks identical
15:05 mwhahaha not sure then, you could try make deep_clean
15:05 Guest21752 hm ok
15:05 Guest21752 so if i made it this far in the build nothing should be wrong with my ubuntu mirror right?
15:06 Guest21752 or docker mirror?
15:07 mwhahaha deep clean will purge the local mirror
15:07 mwhahaha but not sure given issues you're hitting
15:16 Liuqing joined #fuel
15:23 gcmacmalloc joined #fuel
15:32 elo joined #fuel
15:38 kelepirci joined #fuel
15:44 gcmacmalloc joined #fuel
16:00 fritchie joined #fuel
16:01 vkulanov joined #fuel
16:11 Miouge joined #fuel
16:18 Miouge joined #fuel
16:18 code-R joined #fuel
16:26 Miouge joined #fuel
16:37 v1k0d3n_ joined #fuel
16:39 Miouge joined #fuel
16:42 Miouge joined #fuel
16:42 v1k0d3n joined #fuel
16:44 dteselkin Hi, could anyone from fuel-library plz merge https://review.openstack.org/#/c/348221/ ?
16:50 mwhahaha dteselkin: done
17:04 Miouge joined #fuel
17:05 dteselkin mwhahaha: thank!
17:05 dteselkin * thanks :)
17:06 Kdecherf where is the operation guide of fuel 9.0? it disapeared from docs.mirantis.com
17:07 elopez joined #fuel
17:08 elopez_ joined #fuel
17:16 Miouge joined #fuel
17:22 Miouge joined #fuel
17:35 goldenfri So I am running a deploy in fuel 9 and I'm getting the following error: Deployment has failed. All nodes are finished. Failed tasks: Task[connectivity_tests/10], Task[connectivity_tests/3], Task[connectivity_tests/1], Task[connectivity_tests/5], Task[connectivity_tests/4], Task[connectivity_tests/7], Task[connectivity_tests/6], Task[connectivity_tests/9], Task[connectivity_tests/8] Stopping the deployment process!
17:36 goldenfri in the puppet logs it looks like it can't get to 'http://mirror.fuel-infra.org/mos-repos/ubuntu/9.0/' but I'm not sure why, has anyone seen this?
17:39 goldenfri looks the same as this bug: https://bugs.launchpad.net/fuel/+bug/1568891 but shouldn't that already be in Mirantis Openstack 9?
17:43 Miouge joined #fuel
17:44 mwhahaha goldenfri: that usually means your network configuration is not right
17:44 goldenfri hrm, it verified
17:44 mwhahaha you need to have internet access to deploy so that check occurs after the network configuration has been applied
17:44 mwhahaha so log into one of the systems and make sure you can access that url from the deployed nodes
17:45 goldenfri none of the node deployed, but I can access it from the fuel master
17:45 mwhahaha right but you can ssh to the node from the master
17:45 mwhahaha just cause you can access from the fuel master doesn't mean the nodes can
17:45 goldenfri right
17:45 goldenfri ok, I'll check on that, thanks
17:47 goldenfri yea so dns is working but packets aren't going though, so something is wrong with the fuel node's NATing
17:49 mwhahaha at that point it shouldn't be natting i don't think
17:50 goldenfri I'm not using DVR, so shouldn't they all be going though the fuel master
17:50 mwhahaha no
17:50 goldenfri oh I mean the controllers
17:51 goldenfri ok so controller is working
17:51 goldenfri just not the other nodes
17:51 mwhahaha so i'm not sure about the ordering of that
17:52 mwhahaha it's possible there might be an issues but i don't know how you've configured things
17:53 goldenfri hrm yea I'm pretty sure its the exactly way I configured things using fuel8, which worked
17:53 goldenfri nothing fancy
17:53 goldenfri but yea not verify doesn't work and gives the same problem, can't access the repos on the compute nodes, hrm
17:54 goldenfri The Public IP range should exclude the fuel node IP right?
17:57 v1k0d3n joined #fuel
18:00 mwhahaha so if that configure worked with fuel8 it might be a bug, so it'd be helpful to report it with your configuration so we can try and reproduce it https://wiki.openstack.org/wiki/Fuel/How_to_contribute#Test_and_report_bugs
18:00 fritchie joined #fuel
18:00 goldenfri Ok thanks, I'm just making sure its a bug and not something I screwed up first :)
18:04 goldenfri ok yea, I just reset the envirnment and the compute nodes can get to the outside again, so something funny is happening when I deploy
18:08 mwhahaha well when it's reset it's using the master
18:09 mwhahaha so whatever your network configuration is doesn't allow for internet access to the other nodes
18:09 mwhahaha i'm not sure which network it set as the default route but i thought it was the management network so as long as that has a viable nat it should work
18:15 goldenfri ok, yea I thought it should be the admin that it pxe boots off of, which is valid. So I'll play around with it some more and submit a bug if I can confirm its not working correctly, thanks for your help
18:20 mwhahaha no once the deployment starts we use the fuel master for a bit but when the netconfig task runs it's now your network settings that it uses
18:20 mwhahaha you can get around this internet requirement by using a local mirror on the fuel master
18:31 drmccollum I'm having an issue with the 9.0 master node getting to the internet during the install. Twice now, I have to accept the warning and continue the installation and after the install, run fuelmenu and keep messing with the network settings until the route starts working. I'm 99% sure this isn't an issue with my network because it just starts working after playing around with the networking on fuel.
18:32 mwhahaha what?
18:37 johnavp1989 mwhahaha: any idea why "Ceilometer test to check notifications from Keystone" would be failing on Get trust notifications?
18:39 mwhahaha is that a fuel health test or a tempest test or something?
18:39 johnavp1989 It's a fuel health test
18:41 mwhahaha perhaps there's a missing notifications setting in keystone
18:41 mwhahaha i'm not familar with that test
18:43 jackdastripper are you aware of 9.0 node deployments failing if you select L3 HA?
18:43 mwhahaha nope
18:43 mwhahaha whats the error?
18:44 johnavp1989 mwhahaha: ok thanks. i'll look there
18:44 mwhahaha johnavp1989: if you do find the missing setting, please file a bug so we can get it fixed
18:45 johnavp1989 will do
18:45 jackdastripper I'll have to find it again. I reset my environment to try again and the duplicate hostname bug bit me, so I just now rebuilt my master node.
18:46 jackdastripper I found someone else that submitted a bug with the same error I had, but I didn't see how to fix it.
18:46 mwhahaha https://bugs.launchpad.net/fuel/+bug/1604547 thatone?
18:47 jackdastripper yes, that is the duplicate hostname issue, but I'm also getting failures when I enable L3 HA.
18:47 mwhahaha we couldn't reproduce that bug
18:47 mwhahaha when you get the failures, we need the error message
18:49 jackdastripper I had that exact same error message when I tried to use the same hostname after I removed hosts and retried the deployment. I'll get the error and bug report for the L3 HA once I get the install running again.
18:49 Miouge joined #fuel
18:53 mwhahaha ok. as for the dupe hostname thing, what was the order you did the addition/removal/reset?
18:53 mwhahaha we couldn't reproduce it under normal deploy, remove node, deploy, add node, deploy scenario
18:57 jackdastripper I PXE booted my hosts for the first time after installing master. I set the host name in the node window and hit enter. Click the gear button and set the hostname in the node detail window and hit enter. Deployed changes. After deployment failure. I turned off the nodes and waited until the remove option appeared. I removed the hosts and PXE booted again. When they came up I went through the same naming process and clicked deplo
18:57 jackdastripper error in https://bugs.launchpad.net/fuel/+bug/1604547
18:58 jackdastripper I guess I forgot to add that I set the node roles after I changed the hostnames and before deploying.
19:00 jackdastripper Any chance you could add the option to set the tagged vlan for the master node NICs during installation?
19:01 mwhahaha could create a bug for it but it seems kinda complicated so i'm not sure it's something that can get added to 9
19:04 mwhahaha i'll have to try and reproduce that hostname thing again
19:04 mwhahaha looks like when you force removal it might not be clearing it up correctly
19:04 mwhahaha or something
19:05 jackdastripper yep. I was trying to figure out if I could manually clean up the nailgun database, but I'm just going to snapshot my fuel master before I create the env.
19:18 eric_lopez joined #fuel
19:29 goldenfri So my problem seems to be resolved, I think I was just dumb and included the fuel node in the public network range.
19:33 mwhahaha :D that's be a problem
19:34 goldenfri well crap, I spoke too soon lol, provisionoing goes fine, but deployment just had the same problem again
19:34 mwhahaha what's the default route on the nodes
19:35 goldenfri default         192.168.0.1     0.0.0.0         UG    0      0        0 br-mgmt
19:36 mwhahaha yea so does your management network have internet access?
19:37 goldenfri wouldn't that be NATed though the controller?
19:37 mwhahaha no
19:38 mwhahaha i don't think so at least
19:38 goldenfri Thats how it worked on fuel8
19:38 goldenfri because my conmpute nodes didn't have public IP addresses
19:38 goldenfri or at least I think thats how it worked
19:39 mwhahaha right they don't have public ips
19:40 mwhahaha but the management network is how it's trying to get to the internet (default route)
19:40 goldenfri right and that routes though the controller
19:40 mwhahaha no
19:40 goldenfri how does it go from a private network to the public net then
19:40 mwhahaha not sure let me look at my test env
19:41 goldenfri I also have a stack running fuel 8 and it seems to be set up the same
19:42 mwhahaha so on my 10x env (basically still 9) the default route on my compute is default via 10.109.11.8 dev br-mgmt
19:42 mwhahaha which is my vrouter ip
19:42 mwhahaha which would be what you are describing
19:42 mwhahaha sooo what did you configure :D
19:44 mwhahaha did you check the assign public network to all nodes box?
19:44 mwhahaha mine's unchecked
19:44 goldenfri the management net is 192.168.0.0./24, left the defaults there, and just changed the public net
19:44 goldenfri mine is unchecked as well
19:44 drmccollum What's the deal with fwaas, lbaas and vpnaas plugins. They haven't been updated since 7.0. Are we supposed to use the standard Ubuntu Openstack packages for 8/9?
19:46 mwhahaha goldenfri: can you give me an ip a and ip route from a compute node?
19:46 mwhahaha drmccollum: i don't think we're officially supporting those plugins. i know the lbaas one we are not so you'd be on your own to manually configure them after installation
19:49 goldenfri on a compute 192.168.0.12 is the ip on br-mgmt and the route is: default         192.168.0.1     0.0.0.0         UG    0      0        0 br-mgmt
19:49 mwhahaha is there a 192.168.0.1?
19:50 goldenfri doesn't appear so
19:50 goldenfri that could be a problem
19:50 elopez_ joined #fuel
19:50 drmccollum Do you know of a good article on how to setup those on Ubuntu? I was only able to find stuff for Centos/RHEL for Mitaka.
19:51 mwhahaha goldenfri: i think once the cluster is setup it should proxy via the controllers but i think as part of the install that doesn't happen
19:51 mwhahaha goldenfri: if you don't want ot make a 192.168.0.1 then you could just create a local mirror
19:52 goldenfri what should be 192.168.0.1
19:52 mwhahaha a gateway of some sort on your network :D
19:52 mwhahaha drmccollum: other than the package names, the config file updates should probably be similar
19:54 goldenfri ok, I am super confused now. I didn't have to do anything special on fuel 9
19:54 goldenfri I mean 8
19:55 elopez_ joined #fuel
19:55 goldenfri I don't have a 192.168.0.1 listed on my fuel 8 stack, but that has the same default route and it works just fine
19:55 mwhahaha i think you're hitting a bug around the default gateway setting
19:55 mwhahaha the quickest way for you to get deploying is setting up a local mirror on the fuel master
19:56 goldenfri Ok, I'll try and set that up. Thanks again for your help
19:56 mwhahaha by the way what tasks have run on the compute node?
19:57 goldenfri How can I tell?
19:57 mwhahaha you can grep MODULAR from the puppet log for the node and provide it via paste.openstack.org
20:00 fritchie joined #fuel
20:01 goldenfri http://paste.openstack.org/show/543439/
20:02 mwhahaha ok so it's working as expected
20:02 mwhahaha i guess that's something that has changed between 8 and 9 is the assumption about the management network gateway
20:06 mwhahaha hmm perhaps not, the code seems to be similar to 8's we just moved it
20:11 mwhahaha goldenfri: w3ht does 'hiera network_scheme' return on your compute node
20:13 goldenfri http://paste.openstack.org/show/543441/
20:14 mwhahaha that's weird
20:14 mwhahaha did you check the box to use the whole management cidr?
20:14 goldenfri no
20:14 mwhahaha what about hiera -h network_scheme
20:15 goldenfri http://paste.openstack.org/show/543442/
20:16 mwhahaha you using ironic?
20:16 goldenfri not yet haha
20:16 mwhahaha well that's the difference between what i have and you have :D
20:16 goldenfri ah ok, let me try it with out that
20:16 mwhahaha wonder if that broke the way it calculates the gateway info
20:18 goldenfri I have ironic selected on my fuel8 stack as well and it worked as expected
20:18 mwhahaha so the issue is that the network information there gets calculated by nailgun which may have changed between 8/9 (i'm not sure)
20:18 goldenfri ah ok
20:18 mwhahaha what i see is the gateway getting set to the vrouter like you said it should
20:18 mwhahaha so i'm not sure why in your case that is not happening
20:19 goldenfri ok, well I'll give it a try with out ironic and see if that changes things.
20:22 drmccollum joined #fuel
20:22 jackdastripper joined #fuel
20:26 HeOS joined #fuel
20:32 gcmacmalloc joined #fuel
20:46 DavidRama joined #fuel
21:04 elo joined #fuel
21:15 goldenfri FYI, I got the same error with out ironic.
21:19 mwhahaha weird
21:20 jackdastripper joined #fuel
21:57 v1k0d3n joined #fuel
22:09 omgdragons joined #fuel
22:09 omgdragons hey guys
22:09 omgdragons my fuel-8 deployment says my cloud has 7tb of disk space but the cloud itself says it only has 1.3tb
22:09 omgdragons what am i missing here
22:12 fatdragon joined #fuel
22:24 DavidRama joined #fuel
22:26 xarses omgdragons: It's based on the free space of all of the node roles and doesn't exactly filter based on volumes that can actually store data
22:27 xarses its quite simple in that regard, I think it was worked on and is more representative in 9
22:27 omgdragons xarses: you mean fuel web ui
22:27 xarses yes
22:28 omgdragons it just dawned on me :-O
22:29 omgdragons unless i specify that a volume is of type lvm - it gets created in the virtual storage?
22:32 xarses when you create a instance?
22:32 omgdragons so as a policy i have my guys create a volume from an image and then boot that volume as an instance
22:34 omgdragons right now im at 99% capacity disk-wise. horizon says that 'local storage' (which in fuel node disk config is labeled as 'virtual storage') is at capacity on all of my nodes.
22:34 DavidRama joined #fuel
22:35 omgdragons i can only surmise that this 'local storage' is where all of these volumes are created - unless during creation i specify the type as lvm.
22:36 omgdragons type: volumes_lvm i mean
22:39 * omgdragons freaks out
22:39 omgdragons yep.
23:06 xarses virtual storage in the fuel ui is /var/lib/nova
23:06 xarses on the compute node
23:07 xarses if you boot the instance with ephemeral storage, it will be started here
23:07 omgdragons thats what i had thought
23:07 omgdragons however, if i create volume and do NOT specify type - it is created there as well
23:07 xarses if you use a cinder volume, then it will come from wherever that provider stores its space
23:07 xarses did you make all the compute nodes also cinder nodes?
23:08 omgdragons which is cinder-volume
23:08 omgdragons yes
23:08 omgdragons root@node-15:~# pvs   PV         VG     Fmt  Attr PSize   PFree     /dev/sda4  os     lvm2 a--   54.00g   4.00m   /dev/sda5  cinder lvm2 a--  585.87g 349.87g   /dev/sda6  vm     lvm2 a--  196.73g   4.00m
23:09 xarses afaiu the space should be consumed from the cinder vg then
23:09 xarses not /var/lib/nova
23:09 omgdragons this is compute node
23:09 omgdragons right
23:12 xarses so someone ended up eating all of /var/lib/nova or the image cache did,
23:12 omgdragons xarses: so looks like the space gets consumed from the vm pv unless i specify the type as volumes_lvm
23:13 xarses correct, the default is ephemeral, which will come out of vm /var/lib/nova
23:13 xarses image cache can also end up there
23:13 omgdragons can i migrate these?
23:13 xarses image cache?
23:14 xarses or are you trying to convert the ephemearl instances to cinder?
23:14 omgdragons latter
23:14 xarses I think you can take a snapshot and save that to a volume, but I haven't done anything like that in a while
23:16 omgdragons this, i believe, was broken in the past because it created zero sized snapshots
23:16 omgdragons mkay well not everything is totally terrible so im relatively calm. ish.
23:16 omgdragons thank you for the moral and techincal support, xarses :)
23:17 xarses your welcome
23:17 xarses be careful around here, we have lots of rope and trees
23:17 xarses most people make swings ;)
23:17 omgdragons ahahah
23:18 omgdragons yeah im familiar :)
23:34 jackdastripper joined #fuel

| Channels | #fuel index | Today | | Search | Google Search | Plain-Text | summary